ChatGPT Was Not the Beginning

At the end of 2025, a model with barely four billion parameters appeared on Hugging Face, built by a team at the Faculty of Informatics of ELTE (Eötvös Loránd University). Racka-4B was introduced as the first Hungarian reasoning language model. The announcement rightly drew attention, but anyone who reads only the headline could easily believe that the history of Hungarian language technology is a few years old. It is not. The first Hungarian results in computational linguistics date back more than six decades, to a dictionary from Debrecen and a computer built in Budapest from copied Soviet designs.

This article walks through the seven decades that led from the early machine translation experiments to today's Hungarian large language models. The story is about two things at once: a surprisingly deep and continuous professional tradition, and a sharp break that came precisely in the current LLM era. While the knowledge and data assets of Hungarian language technology accumulated over decades, most of today's leading models are no longer foundation systems developed at home. They are Hungarian adaptations of open global models, Qwen and Llama. The question we return to at the end: is this a new era, or a new technological layer on an old tradition?

The history of machine language processing in Hungary does not begin with a linguistic idea but with a piece of hardware. On 21 January 1959, the M-3 was handed over: the first electronic, von Neumann-architecture computer in Hungary, built by the Cybernetics Research Group of the Hungarian Academy of Sciences (MTA) under Sándor Varga, based on Soviet documentation. The young Bálint Dömölki was among its developers. The "M" in the machine's name, incidentally, does not stand for "magyar" (Hungarian); it abbreviates the Russian "malaya" (small). The date is now celebrated as the day of Hungarian informatics. A few years earlier, László Kozma had already built the MESZ-1 at the Budapest University of Technology (BME), a "zeroth-generation" calculator made of telephone exchange relays, which was not yet a von Neumann machine. So the computer existed. The question was what to use it for.

Linguistics showed up early. In 1962 the MTA Computing Centre launched the journal Computational Linguistics, the first Hungarian periodical in the field, and in 1964 the programmatic volume "A matematikai nyelvészet és a gépi fordítás kérdései" (Questions of mathematical linguistics and machine translation) appeared, edited by László Kalmár and Zsigmond Telegdi. Its authors and speakers included the cream of Hungarian linguistics and mathematics of the time, from György Hell and Sándor Kónyi through György Szépe to Ferenc Kiefer. The volume and the journal reflected the scientific momentum of the Cold War: machine translation was then a large international project, and Hungary joined in early.

This is the place for a first careful qualification, because claims of being "first" will follow us throughout the story. György Hell's research on Russian-Hungarian machine translation was real, but it was theoretical and at the level of subsystems: it dealt with the machine storage of root morphemes and with individual analysis subtasks. It left behind no documented, complete, working translation system. When the notorious American ALPAC report appeared in 1966 and set back machine translation funding worldwide, the effect reached Hungary too: the machine linguistics group of the Computing Centre was turned into a Documentation Linguistics Group. The early momentum broke before it could mature into a complete system.

If one name had to be given as the founder of computational linguistics in Hungary, it would be Ferenc Papp. In the 1960s, the professor at Kossuth Lajos University (KLTE) in Debrecen launched, under what was then called "mathematical linguistics," the line of research that treated the Hungarian language as data, with computers. His main work, A magyar nyelv szóvégmutató szótára (a reverse-order dictionary of Hungarian), was published in 1969, and it was no small undertaking: Papp processed the 58,000 entries of the Explanatory Dictionary (Értelmező Szótár) with machine data analysis and examined the entire noun material of more than 31,000 words. The dictionary sorted words in reverse letter order, which made the system of stems and suffixes easy to survey. The database created this way has been used ever since, converted to newer and newer storage media, for building Hungarian computational dictionaries.

This work is often said to have been one of the world's first computational language dictionaries. Precise wording matters here: the American Brown Corpus was completed in 1961, earlier, so the label "first" holds only in a partial sense, as pioneering for Hungarian or among non-English languages. That does not make Papp's achievement any smaller. Formalizing the morphology of an agglutinative language and putting it on a computer, decades before computing capacity was abundant, in the age of punch cards, established a way of thinking that shaped the whole later history of Hungarian language technology.

Why morphology became the lasting central problem of Hungarian NLP follows from the structure of the language. Hungarian expresses grammatical relations with suffixes attached to the stem, so a single stem can produce hundreds of word forms. A system tailored to English cannot cope with this: what takes a few words in English becomes, in Hungarian, an unsurveyable set of inflected forms. Researchers were therefore forced early on to formalize linguistic structures precisely, for the machine, and this constraint also became the engine of Hungarian expertise.

The commercial peak of this approach was MorphoLogic. The company was founded in 1991 by Gábor Prószéky, László Tihanyi and their colleagues, and over the next quarter century almost every practical product of Hungarian language technology came from there. Helyes-e? ("Is it correct?") was the first widely used Hungarian spell checker, and it was also built into Microsoft Word. The HuMor morphological analyzer-generator became a reusable engine that was later rewritten in open form as emMorph. MoBiMouse, an instant dictionary called up with the mouse, was nominated for the EU IST Prize in 1999, and MetaMorpho was, according to the company, the first working Hungarian machine translation system. HuMor's descriptive formalism was also applied to seven other languages, from Polish to Spanish, and even to small Uralic languages (Komi, Udmurt, Mansi, Nenets, Nganasan). Kilgray, a spin-off of MorphoLogic, achieved international success in translation software with memoQ.

MorphoLogic closed in 2017, and its commercial balance sheet should be treated with caution. But the accumulated knowledge was not lost; it migrated toward the Hungarian Research Centre for Linguistics (NYTK) and Pázmány Péter Catholic University (PPKE). There is also a deeper lesson. The problem of morphological analysis never went away; it only changed its name. What was called morphology in the nineties returns today as the question of tokenization and fertility: a tokenizer optimized for English cuts Hungarian words into many small fragments, which means longer, more expensive and weaker processing. One of the most important adjustments in today's Hungarian LLMs is precisely adapting the tokenizer to Hungarian. It is the same problem Papp and MorphoLogic wrestled with, only in a different technological layer.

The 2000s are the "invisible" but perhaps most important era of Hungarian NLP. This is when the language resources were built without which today's models would be unthinkable. Building a corpus, however, is neither spectacular nor fast. At the University of Szeged, Dóra Csendes, János Csirik, Tibor Gyimóthy and their colleagues created the Szeged Corpus and the Szeged Treebank, still the largest manually annotated Hungarian language database. At the MTA Research Institute for Linguistics, the Hungarian National Corpus, the balanced reference corpus of the language, was built under Tamás Váradi. At BME's Centre for Media Research and Education, the circle around András Kornai produced open tools such as hunmorph, the Hunglish parallel corpus and the Hungarian Webcorpus. Hunspell also comes from here, and today a good share of the world's browsers and office programs use it for spell checking.

This era also built the community. Since 2003, the Hungarian Conference on Computational Linguistics (MSZNY) has been held in Szeged. It has since become the field's annual showcase in Hungary and, as we will see, the venue where the newest models are introduced. Integrated analysis pipelines such as magyarlanc and statistical tools such as HunPos turned Hungarian into a language machines could process well during this period.

Anyone who trains a Hungarian LLM today is an heir to these manually annotated data and open tools, even if the model card says nothing about it.

Alongside text, speech grew its own branch, one that would deserve an article of its own. Hungarian speech research began at BME as early as the seventies under Géza Gordos, later continued through the work of Gábor Olaszy and Géza Németh. This led to the MultiVox and ProfiVox speech synthesizers, and then to the first Hungarian speech recognition databases, thanks to Klára Vicsi and others. Speech technology travelled the same arc on its own path, from rule-based solutions to deep neural models. The story of the large language models that follows, however, is primarily about text.

In the mid-2010s, Hungarian NLP also entered the neural era. word2vec and fastText word embeddings appeared, representing words as dense vectors and making part of linguistic meaning quantifiable. The real turning point came after 2018, with Google's BERT architecture, which brought models that understand text in context. In Hungarian, the key figure of this direction became Dávid Márk Nemeskey. He first built the benchmark corpora for Hungarian language modeling (emLam) and the nine-billion-token Webcorpus 2.0, and then, in 2021, introduced huBERT.

huBERT was the first Hungarian BERT model: a 110-million-parameter contextual encoder trained from scratch on Hungarian text, which outperformed the multilingual mBERT in Hungarian named entity recognition and other understanding tasks. A small but telling detail is that its tokenizer kept the accents, that is, it took the specific features of Hungarian seriously. It is important, however, to be clear about what such a model can do. A BERT-style encoder is for understanding, classification and extraction, not for writing fluent text. huBERT did not chat and did not compose, but it understood, and at the time that alone was a breakthrough. The generative explosion came only afterwards.

2023 was the year of Hungarian generative models. From the workshop of NYTK and the National Laboratory for Digital Heritage, from the circle of Zijian Győző Yang, László Laki and Noémi Ligeti-Nagy, came PULI-GPT-3SX, trained from scratch on the GPT-NeoX architecture on 32 billion exclusively Hungarian words, with 6.7 billion parameters. This was the first Hungarian large language model built entirely from Hungarian data, from nothing. PULI-GPTrio complemented it with a trilingual Hungarian-English-Chinese version. The brand name refers to the Hungarian puli dog.

This is where the question of the "first Hungarian GPT-3" becomes pointed, because there are several competing claims. HILANCO-GPTX, a bilingual model, also with 6.7 billion parameters, from a collaboration between NYTK and the University of Pécs, was announced as the first GPT-3-type Hungarian model. PULI-GPT-3SX, on the other hand, is the first GPT-3 trained from scratch exclusively on Hungarian. Both claims are true, each in its own precisely defined sense. This is the recurring lesson of "firsts" in language technology: it almost always matters whether we mean developed in Hungary or Hungarian-language, trained from scratch or further trained, research prototype or public product, encoder or generative system.

The largest, and also the most instructive, undertaking of the era is OTP's Hungarian language model program. The bank launched the project at the end of 2021, with state support and SambaNova Systems technology, on a dedicated supercomputer costing HUF 7.3 billion, which was handed over in early 2022. The resulting OTP-13B is a 13-billion-parameter bilingual model, trained on the 84-billion-token Webcorpus3 and the same amount of English data. In terms of size, it is the largest Hungarian-focused model, but it is not public. Its weights are not available, and its performance cannot be independently verified. There is something telling in this: the largest Hungarian model is also the most invisible.

From 2024, the direction of Hungarian LLM development changed, and this is one of the central claims of this article. Models trained from scratch were increasingly replaced by Hungarian adaptations of open global foundation models. The first sign was PULI LlumiX 32K: a model built on Llama-2-7B, further trained on Hungarian text, with a 32,000-token context, which already followed instructions, meaning it answered questions rather than just continuing text. SambaNova's SambaLingo-Hungarian (Llama-2 plus 59 billion tokens of Hungarian data) took a similar path, followed in 2025 by PULI-LlumiX-Llama-3.1 and PULI Trio Q, built on Qwen 2.5, the first Hungarian-focused chat models.

Racka-4B, with which we opened the article, fits into this line. The model, whose name comes from the Hungarian phrase "Regionális Adatokon Célzottan Kialakított Alapmodell" (roughly, a foundation model purpose-built on regional data), is based on the reasoning version of Qwen3-4B. It was made through LoRA-based continued pretraining on 160 billion tokens, in joint work by ELTE and DH-LAB. It is a real result: on Hungarian tasks it performs roughly at the level of 8-billion-parameter models twice its size while being faster, and it was chosen as the best paper at the MSZNY 2026 conference. But here too the precise picture is more nuanced than the headline. The reasoning ability of the "first Hungarian reasoning model" is largely inherited from the Qwen3 base model, and its internal chain of thought runs partly in English. Racka did not learn to think from nothing. It adapted an existing capability to Hungarian, skillfully.

Why adaptation rather than training from scratch became the main direction is largely explained by infrastructure. The backbone of Hungarian academic LLM training is the Komondor supercomputer in Debrecen, which went into operation in 2022-2023 and delivers about 6 petaflops with roughly 264 NVIDIA A100 GPUs. That is considerable capacity, but the cards have 40 gigabytes of memory and the bandwidth of their interconnect is limited, which makes training the largest models from scratch difficult. It is no accident that the Racka team chose LoRA and continued training over a from-scratch approach: here the method was shaped by the available hardware. The PULI brand itself has traveled this path, from the from-scratch GPT-3SX to Llama- and Qwen-based adaptations.

A model is only worth as much as we can measure of it, and Hungarian-language evaluation is a story of its own. In 2022, NYTK introduced HuLU (Hungarian Language Understanding), a Hungarian language understanding benchmark modeled on the international GLUE and SuperGLUE, with subtasks such as HuCoPA, HuSST and HuRTE. HuLU primarily measures understanding, and the test labels are deliberately withheld so that models cannot be "trained to the metric."

In the age of generative models, however, other questions arise, and OpenHuEval tries to answer them. It is a 2025 LLM benchmark specific to Hungarian, which tests models on real internet questions, Hungarian proverbs and culture-specific tasks, largely with the "LLM-as-judge" method, that is, scoring by another language model. This is faster and more scalable, but it can also introduce bias. And here it is worth staying sober: benchmarks are threatened both by contamination (when test material ends up in the training data) and by the bias of the judging model. Measuring Hungarian language competence is therefore itself an open research problem, not a settled matter.

One of the recurring words in public discussion around Hungarian LLMs is "sovereignty." But what can it mean for a medium-resource language? If it means complete domestic self-sufficiency (our own foundation model, our own tokenizer, our own infrastructure, with no foreign dependency), then the goal is unrealistic and wasteful. Developing large language models is a global undertaking. For a language community the size of Hungary's, building every foundation model from nothing is not only unaffordable but pointless, when adapting the open Qwen and Llama gives faster and better results. A more sensible reading of sovereignty is not autarky but access, competence and verifiability: that we have our own verified corpus, tokenizer, benchmarks and infrastructure, and that we understand what we are building and how.

From this point of view, the position of Hungarian language technology is both strong and fragile. It is strong because the data and knowledge accumulated since Ferenc Papp are real: without the manually annotated corpora, the open morphological tools, the benchmarks and a working research community, today's models would not exist. And it is worth dispelling a myth here: linguists have not become superfluous in the LLM era. Quite the contrary. Good training data, reliable benchmarks and the evaluation of Hungarian language quality have increased the value of their role. The position is fragile, however, because the largest Hungarian model is closed, the leading models are built on foreign foundations, and computing capacity is scarce.

There is movement on capacity. Alongside Komondor, KIFÜ (the Governmental Agency for IT Development) is planning the 20-petaflops Levente, and from September 2026 a master's program in artificial intelligence starts at eight universities, partly building on this very infrastructure. In other words, the system is paying attention to both the next generation and capacity. The open question is whether this capacity builds our own verifiable language technology, or mainly makes the consumption of global models more efficient. The two are not mutually exclusive, but they are not the same thing either.

The arc of the story is clear in any case. Hungarian language technology did not begin with ChatGPT, nor with PULI: it started with a dictionary from 1969 and a computer copied from Russian designs, and it continues to this day. The continuity, however, does not live on in an unbroken institutional line, but in accumulated data, tools and knowledge. Ferenc Papp's reverse-order dictionary is still used for building Hungarian dictionaries, sixty years, several paradigm shifts and one reasoning model later. It is the longest, and perhaps the most important, Hungarian language data chain.