Outside of English, ChatGPT makes up words, fails logic tests, and can’t do basic information retrieval.

    • GenderNeutralBro@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      That’s an important point you raise. I feel like a big problem with the LLM projects we see today, including ChatGPT, Bard, etc., is that the developers have tunnel vision. Rather than using the LLM as one component of a system with many well-researched traditional algorithms doing what they do best, they want to do everything within the network.

      This makes sense from a research perspective. It doesn’t make sense from an end-product perspective.

      The more I play with LLMs, the more I feel like their true value is as something like “regular expressions on crack”.

      • JWBananas@startrek.website
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        regular expressions on crack

        How does it go?

        “I know! I’ll use an LLM!”

        “Now you have three problems.”

        • GenderNeutralBro@lemmy.sdf.org
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          True. When you have a hammer, everything looks like a nail.

          I haven’t actually implemented this yet, but I’ve been thinking about making a local file search program using an LLM. It would enable me to search for things in ways that are absolutely impossible with language-naive tools.

          Here are a few examples of tasks I have in the past wanted to do, but was not able to:

          • Find references to the board game ‘go’ but please for the love of god do not return every case of the verb ‘go’. Also include passages that refer to the game but don’t mention it by name.

          • Find all references to foods, eating, or meals.

          • Find all the dialog lines of a specific character.

          You can’t do any of that with simple search, or even with regular expressions. You need general language awareness.

          I know that LLMs will not be perfect at these tasks, either (at least not the current ones), but I think they could be quite effective.