Rok Strniša's Blog

  macOS, Ubuntu and Windows are popular operating systems (OS) for desktop users. There are also OSes for mobile phones, cloud servers, etc. Let’s refer to these as “standard OSes”. With the advent of large language models, such as GPT-4 used by ChatGPT, “AI operating systems” (AI OSes) become possible. In this post, I explain what I mean by this, what an architecture of an AI OS could look like, and where I think the next steps are. Background GPT (Generative Pre-trained Transformer) is a large language model (LLM) that is very good at doing one specific task: “finding relevant word sequences (tokens) that follow a particular piece of text.” It is based on the work by Vaswani et al, e.g. Attention Is All You Need . The models are trained on vast amounts of data from many sources and are then fine-tuned to work even better for certain types of tasks, e.g. summarization, translation and answering questions. While the models are massive (tens of GB), their “context window”, i.e. short-te

Large language models (LLMs) have recently taken the world by storm. You are probably most familiar with ChatGPT, an artificial-intelligence (AI) chatbot developed by OpenAI, which uses the language models also developed by OpenAI. These models are called GPT and the latest version is GPT-4. With every new version, the language models are able to accomplish more complex tasks. For example, they can summarize large texts, create new ones based on complex requirements, write code, give suggestions about anything and now even do your taxes. It’s worth noting that these language models can and do make mistakes. First, they are trained on imperfect data, the model itself may have remembered some details incorrectly (similar to JPEG encoding artifacts), and the instructions given by the user may be ambiguous or misleading. Each of these can lead to incorrect output; however, based on my experience, if the user specifies the task well, mistakes are already quite rare. The Idea: Empowering Cha