.
Optimization techniques and transfer learning can be employed to make transformer models more practical for edge deployment.
Chamath has started a startup named 8090 which will develop 80% of the features of a SAAS and sell at 90% discount. He claimed that he can build many current SAAS software with AI and open source tools at very low cost, with few people and very fast. He implied using contractors in India.
The key is building the right 80% of features. Most features are used very little. I don’t think he could achieve 90% discount though. The gross margins aren’t high enough, and he’d still have to host in public clouds. Features that aren’t used often don’t add much cost, so eliminating them doesn’t save as much as he’s portraying.
1 Like
.
My memory failed me. He did said is the most useful 80% and articulate how he plans to figure that out.
Is what doubters said. He is an entrepreneur, he knows what non-entrepreneurs don’t know and can get done what others claim can’t do. Ofc time will tell he is that good 
The inconvenient truth is that different people use different 80% of the features.
.
Implied no need to capture 100% of original market.
Um, what has he done? He seems to have lucked up despite failing at most things he did.
If this is accurate, then OpenAI is already disrupted. It’s 1/10th the cost. It would also mean people need far fewer GPUs.
They are doing what I suggested. Instead of running one model that tries to solve everything, they evaluate the task and run the best model for it. It lets them run a smaller model for each task which is more cost effective without decreasing accuracy.
Databricks is already supporting it.
I found out about it, since we are going to do some testing with it.
I tested for a specific text generation task. ChatGPT is miles ahead of mistral(mistral 7b and mistral 8x7b). In fact, for mistral I had to do more prompt engineering and give multi shot inferencing. While ChatGPT gave better output without any of it.
Let me know how it turned for you guys.
2 Likes
Nobody wants to say a word on Rabbit R1?
In 2024, many companies are using AI as a reason (aka excuse) to layoff slackers and cutting layers of bureaucracy. If did well should boost the bottom line. Guess those on the ground know which ones would likely be successful.
There’s so much that can be done with far less sophisticated tools than AI. It’d accelerate a ton of companies stopped rewarding team size with promotions and titles.
Everyone was bitching about accruals. Without any AI, I was able to work with IT and accounting to automate 80% of them. We also switched to an API to close POs vs procurement manually closing them.
Coding is dead.
Apple Watch will obsolete all standalone edge AI device once Apple introduces an Gen AI/LLM enabled Siri.
The amount of electricity to power that is insane. I’m not sure about the power requirements on the non-H100’s. If they are similar, then powering all 600k is nearing the annual power output of the Hoover Dam.
The other issue is existing data centers aren’t built for that type of power draw. You can only fit 2 per rack due to power limits, or you leave a lot of empty space.
2 Likes
Each H100 sells for around $30K.
So Zuck’s 600K H100’s cost $18B. I am sure Zuck has deep volume discount from Jensen but even if it’s half off we are talking about around $10B. Just for GPU’s. You also need to build or modify datacenters to fit them, and maybe a power plant or two to power them.
A little ago it was reported Apple is going to spend “up to” $1B on AI. Now we know how pathetic that amount is compared to Zuck.
Digital Ocean’s GPU price chart. H100 costs $2.24 an hour if you sign a 3 year contract. Otherwise it’s $6 an hour.
350k H100s + other GPUs = 600k H100s equivalent
Or super efficient. Anyhoo, Apple has less DCs than META.
If something is equivalent to H100 they will cost Zuck roughly the same price. He has enough volume to negotiate price down. So that math is already baked into the discounted price.
That’s a myth. Just try Siri and see how bad it still is, even though it was first to the market.