Re: ChatGPT
Posted: December 8 23, 1:37 pm
I have not played around a ton with LLMs, but I have a bit and I think my perspective is more middle of the road.
I do think they are going to make a big difference, I think things like product 'wizards' and other things that historically are a simple flowchart masquerading as deep customization are going to be dramatically improved. Many synthesis tasks will be a lot faster too. For example I was asked to write a recommendation letter for a friend, and I thought of roughly what I wanted to communicate, but I struggled to turn it into the format/paragraphs that would be an acceptable end product. Asking ChatGPT to take my bullet points and turn it into a draft I could then tweak for my voice/preferences was much faster and used a lot less brainpower.
Where I think some of the hype is overblown is around certainty. Counter-intuitively, I found ChatGPT struggles a lot with math and computation. At its core, my understanding is that LLM's are trying to respond with something that sounds like the answer. It's obviously much more complex than that simplification, but if you keep that in mind it helps explain why it can pattern match and combine information in an impressive way but fail to multiply two numbers that a solar calculator handles 100% of the time. It also struggles with consistency, you can give it identical or near identical instructions and receive dramatically different results. This is more vibes-based, but I also feel that with some of the AI use cases or products we've let ourselves get carried away based on trailers and best-case demos v. real-world experience. This technology is going to change the world, but I think the speed of that change can be oversold if you're watching manicured demos and extrapolating speed based on that demo being a solved problem in most/all cases. A good comparison might be that for all the annual attention their seemingly annual viral video gets, we all still don't have Boston Dynamics robots running around doing tasks for us.
That said, maybe I underestimate the speed of improvement, there's a lot of investment and interest so it's possible that in a couple years my current issues may seem silly. But I think that's borne out less in how much people can wow you by putting the technology's best foot forward, than by seeing durable product/process improvements that play out in real life.
I do think they are going to make a big difference, I think things like product 'wizards' and other things that historically are a simple flowchart masquerading as deep customization are going to be dramatically improved. Many synthesis tasks will be a lot faster too. For example I was asked to write a recommendation letter for a friend, and I thought of roughly what I wanted to communicate, but I struggled to turn it into the format/paragraphs that would be an acceptable end product. Asking ChatGPT to take my bullet points and turn it into a draft I could then tweak for my voice/preferences was much faster and used a lot less brainpower.
Where I think some of the hype is overblown is around certainty. Counter-intuitively, I found ChatGPT struggles a lot with math and computation. At its core, my understanding is that LLM's are trying to respond with something that sounds like the answer. It's obviously much more complex than that simplification, but if you keep that in mind it helps explain why it can pattern match and combine information in an impressive way but fail to multiply two numbers that a solar calculator handles 100% of the time. It also struggles with consistency, you can give it identical or near identical instructions and receive dramatically different results. This is more vibes-based, but I also feel that with some of the AI use cases or products we've let ourselves get carried away based on trailers and best-case demos v. real-world experience. This technology is going to change the world, but I think the speed of that change can be oversold if you're watching manicured demos and extrapolating speed based on that demo being a solved problem in most/all cases. A good comparison might be that for all the annual attention their seemingly annual viral video gets, we all still don't have Boston Dynamics robots running around doing tasks for us.
That said, maybe I underestimate the speed of improvement, there's a lot of investment and interest so it's possible that in a couple years my current issues may seem silly. But I think that's borne out less in how much people can wow you by putting the technology's best foot forward, than by seeing durable product/process improvements that play out in real life.
