They claim to be better than the gpt-4. So Ruben did 4 tests:
Original author: @RubenHssd
Test #1 → Copying the UI of the website
Test #2 → Write a Linkedin Post
Test #3 → Test their PDF Vision
Test #4→ Large Marketing Tips
Test 1: Copying the UI
Test 2: Write a Linkedin post
This article is about the future of blockchain + royalties.
Claude 3：
Interesting task.
Longer than usual.
There is no title format.
GPT-4：
I really hate their emoticons.
It’s so long, it’s crazy.
It feels like my theme is more complete.
Test 3: Test their PDF capabilities
This is actually a draw.
PDF is highly technical and contains designs, diagrams, and text that can be retrieved from images.
But if I had to award a medal to someone, it would still be ChatGPT because it was slightly more detailed.
That’s all, original author: @RubenHssd
Anthropic is so awesome. Two things released by Claude-3:
-
Benchmark for domain experts. I’m not that interested in saturated MMLU and HumanEval. Claude specifically selected finance, medicine and philosophy as expert areas and reported on performance. I recommend that all LLM model cards follow this so that different downstream applications will know what to expect.
-
Analysis of rejection rate. LLMs’overly cautious answers to innocent questions is becoming an epidemic. Anthropic is usually on the extreme security end, but they recognize the problem and highlight their efforts in this regard. Great!
AWS is already online:
https://aws.amazon.com/cn/blogs/china/anthropics-claude-3-haiku-model-is-now-available-in-amazon-bedrock/
New video:
