The software reports - 'You have a completely different engine here.'

My GPU isn't the one I thought it was. Also it seems some software (like Ollama) really doesn't want a bar of me at the moment for some reason..

But hey, at least my Goblin looks good!

So to put things into context here, today I asked Perplexity to suggest a Language Model A.I that would work on an old school system such as mine, complete with it's i7 2600, 16gb ram and ancient K2000 GPU. And that's exactly what it did, coming up with the combo of LM Studio and a model that works well with older specs 

And so I downloaded it and the model (in this case it was SmolLM2-1.72B-Instruct-Q4_K_M.gguf whatever that means. Apparently this works well with the 2600 because it has AVX2 which the i7 2600 supports. Remember this, it's going to be my undoing in a second.) 

It turns out I didn't have to download the model itself, I could have done this in LM Studio itself. Which I ended up doing and then using the model I downloaded because the first one didn't seem to work. 

The trouble I hit here was that no matter what I tried downloading into LM studio, it'd start at 0.00% and just do nothing about it. I went upstairs and did some gardening, still zero. I painted up the Goblin figure I'm going to include on the motherboard (representing Mikey the Mazda Mechanic obviously), came back and still 0.00%. At one stage after pressing a few buttons I had three things trying to install this model and none of them ventured past the zero stage.

And so when hitting the wall (I know this is going to happen many many times during this project) I consulted the internet and Reddit forums. And on those forums I ended up in a settings section where I couldn't get the Reddit member's proposed solution working because I just found out why nothing was working. 

Namely I have the wrong engine.

Wait a minute here..

Not only could it not recognize my GPU (Can't recognize it, what do you mean? It's right there!) but it was not about to be in the business of doing anything with a CPU that doesn't have AVX2. And all this time I thought there was a 2600 lying under all that cooling.. I must have swapped it out at one stage...but where is it now?

THE PARTS CAME IN, SOME PARTS WENT OUT

This board originally came with the 2600 when I got it for free from the boss's son. And if memory serves, it was still working back then. But when I used it to build a computer for my kids, I swapped out the 2600 for the 3770 for some reason (I have no idea why) and then put it...er...no idea where it would be. And since I bought the computer a couple of years back and we've moved since, well I still have no idea where it might be.

Annoyingly, the i7 3770 is the slightly better of the two options. It's just that it's AVX only and AVX 2 would be much better.

But then the plot thickens because when asking about the i7 2600 and it's AVX2 capability, I'm suddenly told: 

THIS IS WHY YOU HAVE TO CHECK A.I'S ANSWERS SOME ALL THE TIME..

I put it back onto Perplexity, 'Hey can you check this?' and lo and behold: 

Extremely unhelpful Perplexity, thanks so much!

 Luckily it did offer to suggest an AVX version that I might like to try. So I asked for it and if it'd work with LM Studio or if I'd have to go back to the plodding speed of the current model on Anything LM.

LONG STORY SHORT..

Trying to find llama-bXXXX-bin-win-avx-x64.zip like Perplexity suggested was like trying to find a needle in a haystack as there's 573 pages to sift through and search was certainly no help. In the end I gave up looking and instead installed Cuda 11 for my K2000 in the hope that's what LMS needed to at least through the limited Vram into the mix. 

Once that was done I went back to LM Studio to see if things had changed and no, it still hated my CPU and couldn't find my GPU. And so just out of curiosity I decided to download a couple of random models just to see if I could get something loaded. Anything at all really. 
And nothing loaded. 
Not even a small model. 

All stuck on 0.00%

It doesn't work with the file I want, it's decided it doesn't want to work with anything I throw at it. 

SO A QUICK TEST WITH ANYTHING LM AGAIN..

'I wonder if Anything LM will run the SmolLM2-1.72B-Instruct-Q4_K_M.gguf model I downloaded earlier? And yes, yes it did. Without the slightest hesitation or even a 'Waaaaaaah, I can't do this with your choice of CPU and missing GPU here!'

And not only did it load it but once loaded, it hooned along! (Well in comparison to the slooow speeds of the previous model) So now I'm thinking of going back to Anything LM but with this new model as the next test for Mikey The Mazda Mechanic.

And as for LM Studio? Well until a new version comes out that wants to play nice with my setup, I've pretty much washed my hands of it at this stage. Still, worth a try..

TO END: THE GOBLIN

I zink I ze the problem!

I've had this Goblin Mechanic 3d print file on my computer for a while now but for this build, I actually printed him at 200% so he fits nicely on the CPU cooler, ready to whack anything that doesn't want to behave (like LM Studio I guess) with his Fixit Tool. I have no idea what that random wire belonged to previously but it fits Mikey here perfectly.

So at least Mikey was a win today!