[deleted by user]
I’m overall still skeptical, but this does sound a lot more like how I imagine a true AI would work. I’ve also thought LLMs were a dead end for a while now.
The fact that AMI is European will be attractive to many people
Good luck getting your model to learn how to code through physical experience instead of through text.
[deleted by user]
I dunno, the I-JEPA paper only dealt with image classification, and it looks like it isn’t scaling with larger model sizes like the other techniques.
Besides, Meta was one of the biggest failures in AI model building while he was there. Not exactly a confidence booster.
I’m extremely skeptical if he’s truly raising money off of name recognition alone instead of a real demo frontier model that just needs scaling.
[deleted by user]
I’m skeptical, but it makes a lot more sense. You don’t just “learn to code.” Writing the text is the easy part. It’s about solving problems. This is next to impossible to do reasonably without actually understanding what the solution needs to do and what capabilities you have to do it. That’s why the LLM method has produced such shit code. It’s just reproducing text. It doesn’t actually understand the problem or what it can use to get it done.
Coding is a solved problem; people with zero understanding can do it by copypasta from stack overflow, and similarly skilled LLMs can do it right now, cheaper. If you’re a “coder”, you have a lovely hobby but no career. Sorry.
If you’re a software engineer though, you have nothing to fear from current LLMs. But there is much more chance of LeCun’s models learning engineering - i.e. problem solving, in which writing code is just one of the tools, and not even the most important one - through physical experience and not just text. It is, after all, how all the software engineers today did the vast majority of their learning.
LLMs are an obvious dead end when it comes to actual “intelligence” or understanding how the world works.
But, this sounds like a “draw the rest of the owl” situation.
“JEPA learns abstract representations of how the world works, ignoring unpredictable surface detail.”
Oh, it’s that simple is it? Just have it “learn abstract representations of how the world works”. Amazing how nobody thought to do that before!
I think I understand the distinction they’re trying to draw. Current models are trained on billions of pictures of cats and billions of pictures of dogs. You feed it an image of Fido and it finds a point in 2500 dimensional space and knows whether that point is in the “cat space” or “dog space”. It can be very good, but it doesn’t have any “understanding” of what makes something a cat vs. a dog. Humans, OTOH, aren’t trained on billions of images. But, they learn about things like “teeth” and “whiskers” and “snouts” and “eyes”. Within their knowledge of eyes, they spot that vertical slit pupils are unusual and different, and part of what makes something “catlike”. AFAIK, nobody has ever managed to create a system that learns abstract features without intensive human training.
I like that they’re trying something new. But, are they counting on a massive breakthrough on a problem that has existed since people first started theorizing about AI? Or, is it just a matter of refining a known process?
[deleted by user]
It’s pretty crazy to me that zuck let an actual academic like Yann LeCun go for a kid like Alex Wang. Seems like some very short term thinking.
Yann is the annoying nerd that tells you the truth. Alex is the cool kid that tells you what you want to hear.
To be fair, the financial market is deeply rewarding the “tell us what we want to hear” approach.
Even if the time should come where the chickens come home to roost, the key players will have gotten billions out of the mania in the meantime.
So on one hand you have someone making a fair pessimistic assessment of current approaches that isn’t attractive to investors and his suggestion is very unproven. On the other hand you have someone that agrees with whatever the investors want to believe. The latter is, in this situation, an easy payday.
Still gonna use it to enslave and possibly enable a culling of the world population.
deleted by creator
I believe only in success of AI systems based on real neurons(living tissue), not just “the models”. The problem with all current AI system is that they are just modelling how real AI would look and behave like. I appreciate his attempts to turn AI slop into something more meaningful, but I do not comprehend how he is going to achieve this without creating some completely new and revolutionary approach to resemble neurons in computers.
We are not modelling real neurons even. What we have are just big functions with lots of parameters that calculate the output number based on input. That’s all.
[deleted by user]
They don’t even have state in the weights blob. It’s all tokens in an input vector.
What exactly is this for? I understand LLMs have there limits with understanding physical reality, but at least they have a use case of theoretically automating the “symbolic work” ie moving symbols around on a screen or piece of paper, that white collar workers do.
Yes it’ll never be able to cook a meal or change a lightbulb, but neither will this without a significant enhancement in robotics to embody this AI. What’s the use case? Being able to better tell you how to throw a ball then a person?
[deleted by user]
[deleted]
This page is broken. I accepted the cookies and instead of letting me read the article it shows me a full page about cookies that I can’t close.
[deleted by user]
The Wayback Machine has not archived that URL.
[deleted by user]
