I have avoided AI (LLMS) because it didn’t seem nearly useful enough to warrant supporting these tech assholes. Is there anyway to use it locally on a normal device (16 Gb ram, 8 year old Geforce graphics, etc?). Can I just slow it down and get it to work locally?

  • BeefAndPoultry@lemmus.org
    link
    fedilink
    English
    arrow-up
    0
    ·
    15 days ago

    [email protected]

    Which model GPU and how much VRAM? 16GB RAM limits your options a bit but maybe LFM 2.5 24B-A2B

    If not, Gemma 4 e4b or Ling 3.0 Tiny are about as small and fast as you can get while still being decently smart.