• teawrecks@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 month ago

      It could be a virus.

      Even if the people who made it didn’t intend to, an AI writing a bunch of code that (obviously) isn’t being vetted beyond “does it run” could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.

      This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.

        • story@lemmy.zip
          link
          fedilink
          English
          arrow-up
          0
          ·
          1 month ago

          are you on the openbsd mailing list? the reason their code is better has a lot less to to with a lack of AI-generated code and a lot more to do with rigor. I’ve been watching them for a while now, they accept whatever code is coincise that meets their exacting standards

          They don’t have the ability to detect if you’re using AI to code, no one does. they only know if you’re vibe coding, because they are incredibly experienced and intelligent, and they are pretty polite at the people who obviously are, more polite than i would be.

    • jtrek@startrek.website
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 month ago

      Along the same lines as the other guy said.

      It’s difficult to verify that the software does the desired thing in all cases. It’s also difficult to verify that it’s not also a keylogger or other unwanted thing.

      It may also have some AI generated code path that, say, nukes your home directory because the AI determined that was the appropriate error handling. Maybe no one reviewed it so they don’t even know it’s there. It might not even be malicious.

      That’s all true of any software, but rapidly created AI generated code for legally dubious efforts, by unknown agents (no pun intended) is real hecking risky.