As the name suggests, this is Meta’s second version of the tool (LLaMA stands for Large Language Model Meta AI). According to Meta, the new LlaMa was trained on 40% more data than its predecessor and has double the context length.
But how does it compare to some of the other text-generating A.I. tools out there, like ChatGPT, Bing Chat or Google Bard?
I played around with LLaMa 2 to see how it performs on some of the common tasks that generative A.I. tools are useful for. What I found was a powerful open-source model that offers lots of potential to be adapted and customized for different experiences. But as an out-of-the-box consumer-facing A.I. assistant for jobs like writing or researching, LLaMA 2 is a usable, but not superior tool compared to some of the existing bots.
A different kind of A.I. bot
One thing to understand about LLaMa 2 is that its primary purpose isn’t to be a chatbot. LLaMa 2 is a general LLM available for developers to download and customize, part of Meta CEO Mark Zuckerberg’s plan to improve and advance the model.
That means that if you want to use LLaMa 2 as a chatbot, you’ll need to use special demo versions available on platforms like Hugging Face. The version that we used, HuggingChat, was created by the developer community by deploying LLaMa 2 to Hugging Face. There are other available places to try different LLaMa 2-based chatbots, but HuggingChat is a specialized chatbot, created to be an open-source alternative to ChatGPT.
To try HuggingChat click here.
Philipp Schmid, a technical director of Hugging Face, told Fortune that while the chatbot is comparable to other A.I. bots, it’s not a perfect comparison. LLaMa 2’s specialty is that it can inexpensively be shaped for specific needs. The model hasn’t been fine-tuned to a specific purpose the way a product like Bing Chat has.
LLaMA 2 is also not connected to the internet. That means it has a “knowledge cutoff” at December 2022. That’s more recent than the September 2021 cutoff of ChatGPT. The creators of the HuggingChat chatbot added an option to search the web, but it’s still in the early stages and doesn’t give LLaMa 2 the same capacity as other web-searching chatbots. If you need the most up-to-date information from the internet, you’re better served with a tool like Bing Chat or Google Bard.
In a paper announcing the release of LLaMa 2, Facebook researchers wrote that LLaMA 2 models generally perform better than existing open-source models and are close behind closed-source models like ChatGPT, according to the human evaluations in the paper. The paper acknowledges it can’t yet fully compare to GPT4, OpenAI’s most advanced LLM.
Putting LLaMa 2 to the test
I asked the bot to write an email to my co-workers telling them I was going out of town. It spit out a decent memo suited for the crisp formality of the corporate space.
It can write emails, but can it navigate the touchy subjects, like turning down a job offer? I prompted the bot to draft me an email response saying that I couldn’t accept the job offer. It wrote a short, impersonal three paragraphs that might pass as human, but the email certainly wouldn’t smooth any frustrations that would come from receiving a rejection.
So I asked it again, and I requested it to be more specific, personal and apologetic. It responded with a wordy, possibly too formal email, but this time the email was do-able. It seems the LLaMa 2 demo can fake some contrition when requested
LLaMa could do these kinds of tasks, especially if you prompt it with specifics. It could write decent summaries, and it could easily draft a memo should someone need help. There’s functionality for politely declining a meeting — just feed it the specific names, times and reasons — or for writing specific, formal emails.
Compared to ChatGPT, I found LLaMa 2’s penmanship to be decent but overly formal. I’d use ChatGPT because it often has a stronger flair for putting a degree of human in its language. LLaMa was a bit unpolished and generic for these tasks.
For more creative or “literary” writing tasks, LLaMa 2 was mixed. It struggles to follow word count instructions. If I asked for a 150-word short story, it would give me 190 words. It could write a haiku or 16-line poem about any suggested topic, but whether it was any good is hard to say. Do you think “Circuits hum with life, Processors pace the digital strife, Binary symphony” is a strong haiku?
I asked it to write about “the plight of journalism in 2020,” and it wrote a fairly terrible 16-line poem. While the chatbots aren’t known for their literary elegance (and I’m likely not qualified to judge a poem), this poem felt half-baked. It didn’t rhyme, and even though it generated fun lines like “ink-stained wretches, once the fourth estate’s pride” and had a coherent theme, I wouldn’t call it well-written by any stretch.
When it comes to research, LLaMA 2 isn’t up to par
I also quized the bot on some hard facts, asking it to tell me about the property crisis in China. It served up a slew of bullet points summarizing the market, societal problems and infrastructure in China. When pressed for more information, it could even elaborate on the housing prices and the effects of the COVID-19 pandemic.
Then, I asked it to give me a 50-word summary with citations. It gave me 71 words with the names of publications in parentheses at the end. I turned on the “Search web” function, which allows it to pull from the web, and asked again. It gave me 50 words this time, but each link led to a non-existent page.
When asked about what’s going on with the crisis in July 2023, it again fed me a slew of confusing apologies for misinformation and more broken links.
Between the knowledge cutoff of December 2022 and its faulty search function, it’s likely best to not use this for important research. It’s still a demo, but it’s in need of some fine-tuning. The same rules apply to all generative A.I. tools — always do your research about what it creates. But it is especially important to do that for this tool. It hallucinated citations and it has a knowledge cutoff. If I asked it to summarize and condense information or to alter the text, its response would be increasingly prone to hallucinating fake information.
Should you ride the LLaMA?
The LLaMA 2 demo on Hugging Face isn’t the same as the other chatbots like ChatGPT, Google Bard, and Bing Chat. It shows promise for an early version of a chatbot, but it’s still pretty unpolished. It’s not great for researching, and it had some “deceitful” moments (if you’ll excuse the anthropomorphism).
If I were looking to use the demo for anything more than testing and writing memos, I would have to sift through wordy, occasionally unfinished work.
That said, there are countless reasons to use an A.I. chatbot, and tools like the LLama 2-based HuggingChat are constantly being tweaked and updated. So I encourage you to take this bot for a spin yourself, and see if it’s better suited for what you need. Just be aware of its limitations.
You can try HuggingChat here.
And here’s Fortune’s 3-step guide on how to use chat A.I. tools.
In a paper announcing the release of LLaMa 2, Facebook researchers wrote that LLaMA 2 models generally perform better than existing open-source models and are close behind closed-source models like ChatGPT, according to the human evaluations in the paper.What is the llama 2 meta? ›
Llama 2 is a commercial version of Meta's open source AI language model launched in July, distributed by Microsoft's (MSFT.O) cloud Azure services to compete with OpenAI's ChatGPT and Google's (GOOGL.O) Bard in the nascent generative AI market.Why is Llama 2 a big deal? ›
Llama 2's performance is comparable to GPT-4 and Claude-2 and can solve many data security problems that organizations often encounter when entrusting sensitive information to third-party AI hosting companies.What is llama by meta? ›
LLaMA (Large Language Model Meta AI) is a family of large language models (LLMs), released by Meta AI starting in February 2023. For the first version of LLaMa, four model sizes were trained: 7, 13, 33 and 65 billion parameters.Is llama 2 better? ›
Compared to Llama v1 and other open-source models, Llama 2 shows better performance in all benchmarks. Especially in the important Massive Multi-Task Language Understanding (MMLU) benchmark, Llama clearly outperforms its predecessor and the open-source competition.Is llama 2 better than ChatGPT? ›
In a paper announcing the release of LLaMa 2, Facebook researchers wrote that LLaMA 2 models generally perform better than existing open-source models and are close behind closed-source models like ChatGPT, according to the human evaluations in the paper.What is llama 2 trained on? ›
Llama 2 was pre-trained on 2 trillion tokens of data from publicly available sources. The tuned models are intended for assistant-like chat, whereas pre-trained models can be adapted for a variety of natural language generation tasks.Can llama generate code? ›
Trained on a mix of publicly available data, Meta claims that Llama 2's performance improves significantly over the previous generation of Llama models. Llama 2 is the follow-up to Llama — a collection of models that could generate text and code in response to prompts, comparable to other chatbot-like systems.Can you use llama for commercial use? ›
Takeaways. Today, we're introducing the availability of Llama 2, the next generation of our open source large language model. Llama 2 is free for research and commercial use. Microsoft and Meta are expanding their longstanding partnership, with Microsoft as the preferred partner for Llama 2.What age is Llama Llama for? ›
Common Sense Media recommends Llama Llama for ages 4 and up, so Maggie is right in the sweet spot. For context, a number of episodes take place in a school setting, and Llama Llama and his friends are old enough to play without parental supervision, go to the beach, and have sleepovers.
What is this? Chat GPT is owned and developed by OpenAI, a leading artificial intelligence research and deployment company based in San Francisco that was launched in December 2015. No single person owns OpenAI.Is ChatGPT open-source? ›
However, many people have questioned whether ChatGPT is an open-source application. The answer is that ChatGPT is not open source, as the source code is owned by OpenAI. The ChatGPT application is built on top of OpenAI's GPT-3.5 language module, which is also not open source.What is LLaMA used for? ›
As pack animals, llamas can carry 25-30% of their body weight. Llamas can also be a food source, providing their owners with meat and milk. Llama wool is used for weaving clothes and rope, the skin for leather, the fat for making candles, and the dried excrement for fuel.Is llama 2 free to use? ›
Llama 2 is a large language model that offers significant improvements over its predecessor, which was released in February 2023. While the original Llama was only licensed for research use, Llama-2 is free to use, even for commercial applications, under a new open-source licence.Can you run llama 2 locally? ›
We've been talking a lot about how to run and fine-tune Llama 2 on Replicate. But you can also run Llama locally on your M1/M2 Mac, on Windows, on Linux, or even your phone.How many parameters does the llama 2 have? ›
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.Is Llama2 better than GPT? ›
In a paper announcing the release of LLaMa 2, Facebook researchers wrote that LLaMA 2 models generally perform better than existing open-source models and are close behind closed-source models like ChatGPT, according to the human evaluations in the paper.How does Meta llama work? ›
Like other prominent language models, LLaMA functions by taking a sequence of words as input and predicting the next word, recursively generating text.What is the difference between alpaca and llama Meta? ›
While the LLaMA model is a foundational (or broad) language model that is able to predict the next token (word) based on a given input sequence (sentence), the Alpaca model is a fine-tuned version of the LLaMA model capable of following instructions (which you can think of as ChatGPT behaviour).