I had trouble with two Unsloth quants and had to switch to Bartowski’s quant.

IMO it’s a very good model, not just for coding. It’s also very good as a general model. I might even prefer it to instruct.

Rats. Unfortunately, I didn’t use systemd-boot as my bootloader and instead I used rEFInd. And I didn’t see any rescue options in its settings.

I installed cachyOS precisely because the NVIDIA drivers are pre-compiled and already synced. That is, if they will boot, they will boot together like an orchestra. And if they don’t boot, I’m hoping there’s a way to go in there with a live CD and alter the selected kernel. Actually, I wish there was a way to do that from the boot loader.

So far, CashyOS is working well, but I’ve yet to switch to the Nvidia driver. I will try that today.

I’m installing now

I also have Nvidia.

On the notebook I have split Intel / Nvidia. The live installer failed when I tried to select legacy Nvidia. So I had to install using the default in hopes that limiting to Intel GPU at least installs correctly and from there change the kernel to LTS Nvidia somehow.

What recommendations do you have, given Nvidia, to avoid the bootloader/Nvidia problem you ran into?

This is what I’ve been hoping to read, “almost two years.” But let me ask, how many times in that period have you not been able to fully boot into a working desktop environment?

I’ve not even installed Cachyos but I’ve been reading the wiki. If you haven’t done so already, you may have to install the proprietary driver, non-free. Then, if that doesn’t work, there’s a bunch of stuff in the wiki regarding NVIDIA. Yours appears to be an older NVIDIA card, and does not support the latest open driver.

I like the sky graphic.

Pocketpal is what I run. It works well on Android at least.

https://play.google.com/store/apps/details?id=com.pocketpalai

Not exactly. Digits still uses a Blackwell GPU, only it uses unified RAM as virtual VRAM instead of actual VRAM. The GPU is probably a down clocked Blackwell. Speculation I’ve seen is that these are defective and repurposed Blackwells; good for us. By defective I mean they can’t run at full speed or are projected to have the cracking die problem, etc.

The new $3000 NVidia Digit has 128 GB of fast RAM in an Apple-M4-like unified-memory configuration, reportedly. NVidia claims it is twice as fast as an apple stack at least at inference. Four of these stacked can run a 405B model, again according to NVidia.

In my case I want the graphics power of an GPU and VRAM for other purposes as well. So I’d rather buy a graphics card. But regarding a 90B model, I do wonder if it is possible with two A6000 at 64 GB and a 3 bit quant.

I tried Mistral Nemo 12B instruct this morning. It’s actually quite good. I’d say it’s close to dolphin mistral 8x7B which is a monster in size and very smart, about 45 or 50GB. So I’d say Arli is a good deal Mistral Nemo 12B for 4 or $5 per month and privacy so they claim.

If you don’t mind logging for some questions, you can get access to very good or if not the best models at lmsys.org without monetary cost. Just go to the “Arena”. This is where you contribute with your blind evaluation by voting which of two is better. I often get models like 4o and sonnet 3.5 by Anthropic, google’s best, etc., and at other times many good 70B models. You see two answers at once and vote your favorite between the two. In return, you get “free” access.

Be careful with AMD GPUs as they are not as well supported for local AI. However, support is gaining ground. Some people are doing it but it takes effort and hassle, from what I’ve read.

I know that people are using P40 and P100 GPUs. These are outdated but still work with some software stacks / applications. The P40 GPU, once very cheap for the amount of VRAM, is no longer as cheap as it was probably because folks have been picking them up for inference.

I’m getting a lot done with an NVidia GTX 1080 which only has 8GB VRAM. I can run a quant of dolphin Mixtral 7x8B and it works well enough. It takes minutes to load, almost too long for me, but after that I get 3-5 TPS with some acceptable delay between questions.

I can even run Miqu quants at 2 or 3 bits. It’s super smart even at these low quant levels.

llama 3.1 8B runs great with this 1080 8BG GPU at 4_K_M and also 5 or 6_K_M. I believe I can run gemma 9B f16 at 8 bpw.

I installed it in Linux and it’s headed for a live environment.

Starling looks good so far.

One improvement I’d recommend is to make links visible. They are currently the same color as general text in the chat, black by default. I’d recommend blue.

Interactive Brokers is also my next choice. Although, beware that you have to install Java runtime from Oracle in order to be able to log in to they servers. Java runtime environment has seen many beaches of security in the past, particularly the Internet was still in adolescence. Oracle claims to have solved those but this needs to be verified.

I’m waiting for Schwab mainly because, as it turns out, there is magic there. Namely, our assets are protected from online fraud. I’m sure there are limits to that protection. And that protection has applied to their normal online accounts. Will it apply to API accounts? We will have to reread the fine print when it’s final.

Yes but I’ve read that Schwab will have its own API. I read that within the last two months. I’ve also been told as much by a rep, with disclaimers of course. That was a year ago.

Either way, I expect schwab to have an API. Why else buy TD?

I understood that TDA accounts with API would continue to work. Did yours stop working?

I delayed moving an account to TDA with API because I wanted to wait for the first to settle.

The community can only read the source code, as of yet. All of the source code has been provided by a set of internal developers.

The fact that it is open source means that, if somehow two malware elements have made it into the source code, then someone will eventually report it. But this doesn’t mean that two malware elements cannot be there right now.

These two malware hits on total virus scan should be communicated to the developers.

Locally, an attacker still needs to know your password. A strong password can make it too expensive or impractical to brute force.

Filen.io

Works well so far, is end to end encrypted, open source, and the apps are nice and solid.

Checking out [email protected] I saw very few posts by bots. Mainly saw posts by you. I saw one post coming from alien.top .

What’s interesting is that only posts by bots have any comments. So maybe this could be a good way to get communities started.

Therefore, if it’s okay with the admins at the following community, I’d nominate [email protected]

There’s almost nothing happening there.