Right now, I have to manage which model does what. So I don’t. It’s the companies dime after all, and they clearly value speed over efficiency. But the model I directly talk to doesn’t need to be super high end. It could be well versed in which model is best for what task and farm put all of the work to the right model for the job. Seems like that alone could reduce cost a ton. But then start adding in cheaper local models and the price could fall a ton.
It would also be more accurate. Like if I had a small local model for interacting with a set of apis. It wouldn’t make the mistake of not considering that the api paginates results. The general model makes this mistake over and over.
So I agree this is the direction of the future. Just trying to figure out when we will get there.
This is where things seem to be headed. Local models can be pretrained and crammed full of your work’s information, and opencode and other harnesses can pick the right model for the task ahead of time. I could be wrong but i’ve been predicting a drop in frontier model credit usage especially if they hike token prices again. People will only use it if they have to especially if you can find a way to task a local model with a lot of the grunt work.
Nice. I just wondered how they were coming along.
Right now, I have to manage which model does what. So I don’t. It’s the companies dime after all, and they clearly value speed over efficiency. But the model I directly talk to doesn’t need to be super high end. It could be well versed in which model is best for what task and farm put all of the work to the right model for the job. Seems like that alone could reduce cost a ton. But then start adding in cheaper local models and the price could fall a ton.
It would also be more accurate. Like if I had a small local model for interacting with a set of apis. It wouldn’t make the mistake of not considering that the api paginates results. The general model makes this mistake over and over.
So I agree this is the direction of the future. Just trying to figure out when we will get there.
This is where things seem to be headed. Local models can be pretrained and crammed full of your work’s information, and opencode and other harnesses can pick the right model for the task ahead of time. I could be wrong but i’ve been predicting a drop in frontier model credit usage especially if they hike token prices again. People will only use it if they have to especially if you can find a way to task a local model with a lot of the grunt work.
Here’s a video on it