What is Stanford’s Alpaca AI? ChatGPT-like model created by computer scientists for less than $600 

Alpaca is a small Artificial Intelligence language model based on Meta's LLaMA system. The demo was recently taken down from the internet by researchers at Stanford University due to safety and cost concerns.

Large language models contain hundreds or tens of billions of parameters, and their access is usually restricted to companies that have enough resources to train and run these AIs.

Fast-growing Meta decided to share the code for its famous LLaMA system with a few select researchers. The company wanted to find the cause behind language models generating toxic and false text. They hoped that it would work without researchers needing massive hardware systems.

Hence, Alpaca was born. A group of computer scientists at Stanford University fine-tuned LLaMA into a newer version named Alpaca. This newly made version is an open-source seven-billion-parameter model. As per New Atlas, it cost less than $600 to build.

---

Alpaca has been tuned with more than 50,000 text samples to make it more information-accurate

Alpaca's code was released to the public and it captured the attention of several developers. They managed to get it up and running on Raspberry Pi computers and, surprisingly, a Pixel 6 smartphone.

Standford's researchers spoke about how "instruction-following models" including GPT-3.5, ChatGPT, Claude, and Bing Chat have become "increasingly powerful." The institute's website stated:

"Many users now interact with these models regularly and even use them for work. However, despite their widespread deployment, instruction-following models still have many deficiencies: they can generate false information, propagate social stereotypes, and produce toxic language."

They continued by pointing out that maximum progress can be made if the problems are addressed correctly and if the academic community engages with them.

The researchers spoke about how studies on instructional models in academia have become difficult due to the lack of open-source models "such as OpenAI's text-davinci-003."

Alpaca has been fine-tuned with 50,000 text samples that guide the model and coax it into following specific instructions. It helps to make it work like text-davinci-003. The webpage ran a demo of the AI model and allowed anyone and everyone to interact with it. The LLaMA-based model was taken down soon after due to safety issues and the rising costs of hosting it online.

A spokesperson for Stanford University's Human-centred Artificial Intelligence Institute said in a statement to The Register:

"The original goal of releasing a demo was to disseminate our research in an accessible way We feel that we have mostly achieved this goal, and given the hosting costs and the inadequacies of our content filters, we decided to bring down the demo."

Similar to other language models, the Stanford variant is also prone to providing misinformation - often termed 'hallucinations.' Another common result can be offensive texts. The researchers said:

"Hallucination in particular seems to be a common failure mode for Alpaca, even compared to text-davinci-003."

Some noted that the model failed to give an accurate answer when asked about the capital of Tanzania and instead gave false technical information.

Although the demo has been removed from the internet, the dataset and code that describes the fine-tuning of the model remain available for access on GitHub. Researchers are also in talks to release details on the model's weight.

The website stated:

"Alpaca likely contains many other limitations associated with both the underlying language model and the instruction tuning data. However, we believe that the artifact will still be useful to the community, as it provides a relatively lightweight model that serves as a basis to study important deficiencies."

Artificial Intelligence, like the new Stanford variant, is slowly in the works and is set to come back stronger and more accurate than ever.