• 0 Posts
  • 50 Comments
Joined 4 years ago
Aquileo | cake
Cake day: November 9th, 2022

Aquileo | help-circle














  • dreugeworst@lemmy.mltoMicroblog Memes@lemmy.worldspeak for yourself
    Aquileo | link
    Aquileo | fedilink
    English
    Aquileo | arrow-up
    1
    ·
    10 months ago

    I’m not arguing that AI won’t get better, I’m arguing that the exponential improvements in AI that op was expecting are mostly wishful thinking.

    they could stick to old data only, but then how do you keep growing the dataset by the amounts that have been done recently? that is where a lot of the (diminishing) improvements the last years have come from.

    and it is not at all clear how to apply reinforcement learning for more generic tasks like chatbots, without a clear scoring system like both chess and StarCraft have.


  • dreugeworst@lemmy.mltoMicroblog Memes@lemmy.worldspeak for yourself
    Aquileo | link
    Aquileo | fedilink
    English
    Aquileo | arrow-up
    1
    ·
    10 months ago

    the problem is that ai’s are trained on programs that humans have written. At best the llm architectures it creates will be similar to the state of the art that humans have created at that point.

    however, even more important than the architecture of an ai model is the training data that it is trained on. If we start including ai-generated programs in this data, we will quickly observe model collapse: performance of models tend to get worse as more ai-generated data is included in the training data.

    rather than AIs generating ever smarter new AIs, the more likely result is that we can’t scrape new quality datasets as they’ve all been contaminated with llm-generated data that will only reduce model performance