Open Sourcing Samo — ableneo’s chatbot solution
The onset of Artificial Intelligence is imminent. While some are skeptical or even worried, many are actively preparing for its adoption because various businesses’ operations are about to change sooner rather than later.
One obvious area of AI bloom is chat applications, where ChatGPT has set the tone. ChatGPT is great at discussing generic topics; however, it does not (and should not!) have access to a company’s internal data, documentation, guidelines, and product information. For this purpose, the RAG technique has been developed and is now used worldwide.
In May 2023, our AI team at ableneo has started developing a RAG-like chatbot solution and now, we have decided to open source our application. Samo, as it is called, is capable of learning your internal documents and allows you to chat with them, with just a bit of programming.
Traditional programming vs AI
The main difference between traditional programming and AI is that, in the former, a programmer defines the entire decision-making process. In the case of Artificial Intelligence, we are giving the computer a frame (often called a model, possibly a neural network), a structure used to layer and store the decision-making logic. The model then requires a set of samples, often called training data, where the computer usually learns the decision-making within the pre-set model by itself, guided by the ground truth (a final expected output) we provide it.
Good example of the above is the way chatbot applications were developed prior to the rise of ChatGPT. While there might have been a piece of AI dedicated to categorising a user question, the answer and follow-up process was fully scripted, often leading to a lack of flexibility, inaccurate responses and frustrating user experience.
The incremental improvement of Language models, such as GPT, changes this.
LLMs are changing the business
Machines’ understanding of human language has improved significantly. Looking at one of the most cited language understanding benchmarks, MMLU, the leap across the last couple of years is incredible. Just four years ago, the largest GPT-3 variation scored at 43,9%, while now, it is GPT-4o scoring at 88,7% and Gemini Ultra scoring at 90%.
There is more to it though. High-performing LLM, like GPT is not only a chatbot and its understanding of language makes it a great assistant for multitude of tasks such as document categorization, translation, summarization, scriptwriting, sentiment analysis and more.
Chat with your knowledge
Asking questions and learning from the answers is a fundamental concept of humans transferring knowledge to other human beings. The same concept has been transferred for decades to human-computer interaction just because of how natural it is to us.
There still are areas where LLMs are lacking though. Among them is hallucination, deep domain or even company-specific knowledge and others. The issues above can be tackled by a technique called RAG — Retrieval Augmented Generation.
Within RAG logic, the relevant knowledge (could be the company’s guidelines or the content of the book) is being pre-processed and vectorized using an embedding model. This model ensures that the meaning of each part of the knowledge is captured in a vector of numbers. This allows for the recorded knowledge to be compared against the chatbot user question. This then acts as a search for relevant information that is, in the end, used for the generation of the final answer.
If you are curious about RAG and its evolution, feel free to follow ableneo Technology on Medium, as we are releasing a more technical blog about the evolution of LLMs and RAG in the next couple of days.
Step towards open AI
At ableneo, as a company focusing on enterprise-ready open-source software, we have decided to open source our own battle-hardened RAG solution at https://github.com/ableneo/samo under the Apache 2.0 License.
We know that the world is changing faster than ever before. But we can help you distinguish what makes sense to change and what doesn’t. Using our open-source AI chat is one example of how your company can change where it matters.
The implementation being open-sourced is written in Python. It uses a selection of state-of-the-art software components and document processing techniques to ensure our application is robust enough and collaborates well with a wider set of data sources. One such technique is Parent-Child document chunking and context retrieval paired with the Reciprocal Rank Fusion technique.
The solution consists of 2 applications:
- Ingestion — the process of teaching your chatbot a new set of data. Input is a set of text documents that are being learned and stored by the app.
- Server — application exposing the interface for the live usage of the chatbot. The server allows you to:
– Ask questions and get answers.
– Browse the history of your conversations.
– Provide the AI with feedback, which is possible for each chatbot response.
Samo is by default speaking Slovak, but the whole language and tone of voice is configurable as part of the example config yaml.
For more information, please refer to ableneo’s repository or contact us at info@ableneo.com.
Open Sourcing Samo — ableneo’s chatbot solution was originally published in ableneo Technology on Medium, where people are continuing the conversation by highlighting and responding to this story.