Installing the wise old beast - come back next week when he's awake

If I currently ask my current A.I guru (Perplexity) whether this is a good idea or not, it'd probably suggest turning off the computer, stepping outside and immersing my fingers into the nearest patch of grass. Even LM Studio itself has said 'You know what, this is probably too big to take on you know.'

But what could possibly go wrong with using an oversized model that's probably going to take an hour between answers here if it actually does come up with some decent well thought out answers? It's not like I can't wander off, cook some food and come back after eating to learn what it thinks.

So welcome to my first trial with the beast that is QwQ.

Shhhh, don't startle him!

Okay so the obvious question here - what is Qwen QWQ-32B? From the description from LM Studio: 

Reasoning model from the Qwen family, rivaling DeepSeek R1 on benchmarks.

It's quite the thinker. The trouble is that being quite the reasoning deep thinker (and such a beefy size thinker - clocking in at around 17GB), it would require a decently powered environment to sit and do it's pondering in to return an answer in a decent time. 

But here's me - taking this beast thinker out of the lavish mansion life style it's more than used to when it comes rumination time...and sticking it in a computer environment akin to a beat up 70's caravan parked behind a rather sketchy old folks home, turning it into a very slow responsive beast like Grandad becomes after one two many sherries at a family reunion he didn't want to become a part of. 

Perplexity has suggested with my upstairs setup (I am not even going to even contemplate the thought of trying it on for size on the far more old school Garage Intelligence project downstairs) that I should stick to models of the 7B-8B size and only stretching to 14B size maximum if I have to, but to be prepared for some incredible wait times. 

If you missed the number before, QwQ is 32B. If this works it'll work ever so slowly. If it doesn't, well at least I tried. And who doesn't enjoy some random experimentation on this blog?  

And so through LM studio I did download it, started a new conversation with the new model loading and you'll never guessed what happened... 

My computer hit the brakes

Okay, you probably did guess this. Yes this monster was so large that my rig couldn't even load the model and my computer locked up constantly trying to wrestle it all in. As I suggested above I even went off to cook up some lunch for a bit and came back and it still hadn't loaded it all. Forget the moving locations example earlier, just trying to get this thing to load was like taking a Maclaren F1 engine out and trying to shoehorn it into a 70's Toyota Cressida comfortably, expecting the surrounding wiring loom to be just fine with the new powerplant at the first turn of the key. 

Suffice to say that after a reset I dived in, said goodbye to QwQ and continued on. Perhaps I should have run this idiotic idea by Perplexity to save some time and hard disk space?

Time for a slightly smaller reasoning beast who'd be more comfortable in the caravan!

Gemma to the rescue?

She looks far happier for a start!

'Gemma 4 12B Unified reasoning model with image support' might be slower than the usual 8B models I run being a 12 but it's nowhere near 32B dimensions. It's also a third of the size to download and I was curious to see how the image support went. Does this mean I could take a snapshot of my portfolio now and just give it to Gemma to show what I own and get her to work out which little holding I should add to every time payday rolled around? 

It certainly does! And after trying to feed my portfolio to so many different models in so many different formats, it's so nice to take a couple of snapshots and that's it. Build up a decent prompt (again I had Perplexity help me out here), include your picks and off you go!

This in turn has worked out so much better than just providing tickers (like AAPL for example) and having to double back and correct the model when it gives you news about a completely different company entirely - the name is there, the ticker is there, the symbol is there, it's so much harder to confuse. The model can also see the weight size of each offering and how much I have invested. It's a win win all round in just a couple of pics!

Even better is the fact that Gemma 4 12B shows her thinking which means you can see in real time what she's currently mulling over. Given she's 12B I would have thought it might be a slow thinking process but impressively she churns through her thoughts pretty quickly. 

Finally we may have a winner here..

What I got Gemma to do

So each payday I like to buy more shares, increasing the stake in various holdings in my portfolio. And when this happens, I usually invest in just 5 at a time with a focus on pushing up the lowest value, adding to a growth stock, adding to a dividend stock and boosting up at least one Australian stock (because everything not from here hails from the US, except one lonely Canadian share). This time however I asked Gemma to take a look at what I have, do some thinking and see what she could come up with for my next pay day session. 

Again the focus was on at least one growth, one div share and one Australian offering. 

So how did she go in the end?

Gemma thought on things for over 20 minutes which is not surprising given that she had close to 30 stocks to work with. But she took the prompt and ran with it: 

Thought for 21 minutes 59 seconds

To review your portfolio, I have analyzed the holdings based on their business models, market positions, and how they fulfill your specific requirements (Growth, Dividend, ASX-listed).

Here are my recommendations for the 5 holdings to increase:

 -The growth share she picked was also a dividend share because it was in her mind a primary growth industry being in the energy sector. I'll have to fine tune the prompt so that it's growth but no divs for that category. 

-She picked up CNI as CNR but it didn't matter because she still recognized it as Canadian National Railway anyway so she didn't confuse it for something else. 

-Not only did she list the five but she also gave me current holdings not to increase which I thought was interesting including quite rightly two speculative stocks that are going through a bit of a rough patch at the moment. 

-She either couldn't or didn't work out the weight of each stock based on the numbers on the screen shot. I might have to calculate this separately and see if it makes any changes. (Edit: Neither could Perplexity, at least not correctly. Looks like it's time for an excel spreadsheet formula party!)

-I forgot to take out one stock that I actually bought for my kids and that gets regular attention anyway. 

ROUND 2

I removed the stock that shouldn't be there, tweaked the prompt, crunched some numbers to work out the weight and got Gemma to take another look. Strangely she started thinking but with no visible thought process being shown. 26 minutes later she stopped thinking but didn't provide any insights. So after I returned from shopping I asked again and this time she happily provided her version of thought bubbles and got to work with the new info provided. 

Right up until she hit a 'context length limit reached'. Seems she thought a bit too long here and the whole plan ran out of steam. So there was a bit of a tinkering under the hood to give her a bit more working room. It seemed adding the weights of things really started to tax her whiteboard however because the next two times I tried she hit her limit again and nothing for forthcoming.

ROUND 3

So Gemma has a decently sized 265K context length to work with. The only issue with this is that my computer only has so much Ram and just a handful of VRam from the GPU to work with, so I don't actually have what's needed to reach her full abilities. What I could do though is move the slider in LM Studio far enough to hit my maximum limits...and it turns out that's just what she needed. 

Taking over 47 minutes this time (to be fair I was also playing Rimworld at the time which wouldn't have helped her at all by tying up some of the PC's resources in the background..) she picked her five again to the updated prompt:

-Interestingly while I'd put in the prompt that I wanted 1 growth pick that didn't pay a dividend, she weighed up the pros and cons and suggested one that did that would fit the criteria so much better. It was an energy stock and she argued that the stock provided just what the AI renascence would need going forward.  So I'm running with it for now to see what happens. 

-Again she gave me 5 to not add any more investment dollars too currently but confused ticker SHL for SHY (I own SHL but no SHY). Pointing out the mistake she had a big think about everything again, given than SHL now looked better than she first thought. In the end she thought it looked so good, it now became part of the five!

So yes, I will be acting on her advice

Because I'm not asking her to pick possibly risky new stocks to add to my existing holdings, I asked her which ones to increase at a given time. She's thought plenty (which is great), given me great reasons why to increase the ones she's picked and also given me a heads up on which ones to slow down on. The real test will be a few months down the track to see if she's picking the same ones over and over again or if the market has changed so much so, five different choices will have to be made. 

I'll check back on this post back then but for now I'm very please with Gemma's efforts after some fine tuning. Great model you got going on there Google!