Not-So-Large Language Models: Good Data Overthrows the Goliath by Gennaro S Rodrigues

“Those models are starting to gain traction, primarily on the back of their price performance.” The innovative LLM-to-SLM method enhances the efficiency of SLMs by leveraging the detailed prompt representations encoded by LLMs. This process begins with the LLM encoding the prompt into a comprehensive representation. slm vs llm A projector then adapts this representation to the SLM’s embedding space, allowing the SLM to generate responses autoregressively. To ensure seamless integration, the method replaces or adds LLM representations into SLM embeddings, prioritizing early-stage conditioning to maintain simplicity.

So, the Meta scientists noted in their research, there is a growing need for efficient large language models on mobile devices — a need driven by increasing cloud costs and latency concerns. When smaller models fall short, the hybrid AI model could provide the option to access LLM in the public cloud. This would allow enterprises to keep their data secure within their premises by using domain-specific SLMs, and they could access LLMs in the public cloud when needed. As mobile devices with SOC become more capable, this seems like a more efficient way to distribute generative AI workloads. Another boon to the rise of SLMs has been the emergence of specialized frameworks like llama.cpp. By focusing on performance optimization for CPU inference, llama.cpp – compared with a general-purpose framework like PyTorch ­– enables faster and more efficient execution of Llama-based models on commodity hardware.

Fine-Tune Defender XDR for Cost and Coverage

Microsoft has formed a new team to develop “cheaper generative AI” systems, according to a recent report by The Information. This happens while Microsoft is deeply invested in OpenAI, which sells access to expensive large language models (LLM). Ghodsian used fine-tuning with retrieval augmented generation (RAG) to attain quality responses.

In a previous paper, they introduced a new transformer architecture that removes up to 16% of the parameters from LLMs. And another paper from the university’s researchers presents a technique that can speed up LLM inference by up to 300%. I expect closer collaboration between Microsoft’s GenAI team and ETH Zurich researchers in the future. A recent paper by researchers at Microsoft and ETH Zurich introduces a method that reduces the size of models after training. The technique, called SliceGPT, takes advantage of sparse representations in LLMs to compress the parameters in dense matrices.

What piqued my interest is that the company said it can perform better than models twice its size. After initially forfeiting their advantage in LLMs to OpenAI, Google is aggressively pursuing the SLM opportunity. Back in February, Google introduced Gemma, a new series of small language models designed to be more efficient and user-friendly.

Llama 3 – one of the most capable small language models on your computer

Although not confirmed, GPT-4 is estimated to have about 1.8 trillion parameters. There are now Small Language Models (SLMs) that are “smaller” in size compared to LLMs. SLMs are trained on 10s of billions of parameters, while LLMs are trained on 100s of billions of parameters. They might not have broad contextual information, but they perform very well in their chosen domain.

• Traditional methods primarily revolve around refining these models through extensive training on large datasets and prompt engineering.
• Future versions of the report will evaluate additional AI tools, such as those for summarizing, analyzing, and reasoning with industrial data, to assess the full performance of industrial AI agents.
• There’s a lot of work being put into SLMs at the moment, with surprisingly good results.

This means that the model labels parts of the document and we collect these labels into structured outputs. I recommend trying to use a SLMs where possible rather than defaulting to LLMs for every problem. For example, in resume parsing for job boards, waiting 30+ seconds for an LLM to process a resume is often unacceptable.

Tamika Curry Smith was on the ground to share our commitments around #DEI and #AI. 🚗

At #REAutoUSA, Dipti Vachani, our SVP and GM for Automotive shared how we’re working across the stack to deliver solutions that enable software development from day 1, enabled by standards driven by SOAFEE. This can encourage developers to build generative AI solutions with multimodal capabilities, which can process and generate content across different forms of media, such as text, images, and audio. In summary, transitioning to an intelligent, adaptive design supported by a coordinated ecosystem of LLMs and SLMs is essential to maximize enterprise value. Starting at the bottom, we show these two-way connections to the operational and analytic apps.

Or, at the very least, the infrastructure costs to push this approach to AI further are putting it out of reach for all but a handful. You can foun additiona information about ai customer service and artificial intelligence and NLP. This class of LLM requires a vast amount of computational ChatGPT power and energy, which translates into high operational costs. Training GPT-4 cost at least $100 million, illustrating the financial and resource-heavy nature of these projects.

Model Adaptation

It is crucial to emphasize that the decision between small and large language models hinges on the specific requirements of each task. While large models excel in capturing intricate patterns in diverse data, small models are proving invaluable in scenarios where efficiency, speed, and resource constraints take precedence. The breadth of the capabilities is awe-inspiring, but taming such massive AI models with hundreds of billions of parameters is expensive.

LLaMA-13B outperforms the much larger 175B parameter GPT-3 on most benchmarks while being over 10x smaller. The authors argue that given a target performance level, smaller models trained longer are preferable to larger models for a given compute budget due to better inference efficiency. Phi-2 was trained on 96 Nvidia A100 GPUs with 80 gigabytes of memory for 14 days, which is more than most organizations can afford. This is why for the moment, SLMs will remain the domain of wealthy tech companies that can run expensive experiments, especially since there is no direct path to profitability on such models yet. Given Microsoft’s financial and computational resources, its new team will probably add to the open LLM catalog.

“This paves the way for more widespread adoption of on-device AI,” he told TechNewsWorld. Since Ollama exposes an OpenAI-compatible API endpoint, we can use the standard OpenAI Python client to interact with the model. Running the command ollama ps shows an empty list, since we haven’t downloaded the model yet. Additional considerations include adhering to ethical AI practices by ensuring fairness, accountability, and transparency in your SLM.

It’s also worth mentioning that you can use it in over 30 languages, such as English, German, French, Korean, and Japanese. This relates to what I believe is the single-most powerful capability of this model, i.e., that it excels in optical character recognition (OCR). Enterprises are evaluating the cost aspect of implementing ChatGPT App GenAI solutions more closely now as the initial enthusiasm leads to realist calculations. Other situations might warrant particularly low risk tolerance — think financial documents and “straight-through processing”. This is where extracted information is automatically added to a system without review by a human.

Compared to the Fallback approach, which showed high precision but poor recall, the Categorized method excelled in both metrics. This superior performance translated into more effective inconsistency filtering. While the Vanilla approach exhibited high inconsistency rates, and the Fallback method showed limited improvement, the Categorized approach dramatically reduced inconsistencies to as low as 0.1-1% across all datasets after filtering. The SLM serves as a lightweight, efficient classifier trained to identify potential hallucinations in text.

Microsoft Researchers Combine Small and Large Language Models for Faster, More Accurate Hallucination Detection

The first Cognite Atlas AI™ LLM & SLM Benchmark Report for Industrial Agents will be available to download for free on October 28, 2024. The report will then be regularly published to enable digital transformation leaders to use Gen AI to carry out more complex operations with greater accuracy. A new desktop artificial intelligence app has me rethinking my stance on generative AIs place in my productivity workflow. Each medical specialization (oncology, dermatology, etc.) could have its own SLM that scans and summarizes the latest research from medical journals. For example, a medical journal version frees doctors’ time buried in research papers. Healthcare is a good candidate for SLMs because it uses focused medical data, not the entire contents of millions of miscellaneous articles.

“There’s a prevailing paradigm that ‘bigger is better,’ but this is showing it’s really about how parameters are used,” said Nick DeGiacomo, CEO of Bucephalus, an AI-powered e-commerce supply chain platform based in New York City. The researchers, according to the paper, ran experiments with models, architected differently, having 125 million and 350 million parameters, and found that smaller models prioritizing depth over width enhance model performance. This tutorial covered the essential steps required to run Microsoft Phi-3 SLM on a Nvidia Jetson Orin edge device. In the next part of the series, we will continue building the federated LM application by leveraging this model. My goal is to run an SLM at the edge that can respond to user queries based on the context that the local tools provide.

Enterprises can decide to use existing smaller specialized AI models for their industry or create their own to provide a personalized customer experience. Enterprises that operate in specialized domains, like telcos or healthcare or oil and gas companies, have a laser focus. While they can and do benefit from typical gen AI scenarios and use cases, they would be better served with smaller models. Regarding security, a significant advantage of many SLMs is that they are open source.

“The large language models from OpenAI, Anthropic, and others are often overkill — ‘when all you have is a hammer, everything looks like a nail,’” DeGiacomo said. Let’s provide a self-help guide that any organization, regardless of size, can use to build its own domain-specific small language models. Recent industry research and publications have increasingly underscored the relative ineffectiveness of public LLMs in delivering specialized, context-specific insights. While LLMs excel at general tasks, their performance often falters when applied to niche domains or specific organizational needs.

Marlene Weeden