Text mining · NLP · research & innovation policy

Horizon Lens

Every Horizon Europe project funded so far, read by machine. Paste a proposal abstract and see how it would be classified, which policy goals it touches, and which funded projects it overlaps with. Search the portfolio, watch research themes rise and fall, and check how fair the models are. All scoring runs in your browser on models trained from the public CORDIS dataset.

—projects analysed
—EU contribution covered
—research fields modelled
—near-duplicate pairs found
01 · Screen a proposal

What would an evaluator want to know first?

Field classification against the European Science Vocabulary, climate, AI and digital policy markers, the words that drove the decision, and the closest already-funded projects with an overlap warning.

Loading models…

03 · Portfolio map

The research landscape in two dimensions

Each dot is a project, placed by t-SNE over latent semantic vectors and coloured by one of the topic clusters found with k-means. Labels come from class-based TF-IDF. Hover to read a project.

05 · Overlap detection

Near-duplicate abstracts across the portfolio

All pairs of funded projects whose abstracts have TF-IDF cosine similarity of 0.8 or more, sorted into three kinds. Most are the same grant recorded twice, for example a fellowship listed under two grant IDs with the same host and budget. Next come planned follow-up phases by the same coordinator. Only pairs from different coordinators need a human check.

06 · Evaluation and bias audit

Four models, one temporal split, measured honestly

Trained on projects starting up to 2024, tested on 2025 onwards, the way the model would meet new proposals. The decision threshold is tuned on 2024 only. Confidence intervals come from a paired bootstrap over test projects.

F1 per field, smallest fields first

Plain model A in grey, class-balanced model B in blue. n is the number of training projects.

Accuracy by coordinator region

Micro-F1 of model B. Large gaps would mean proposals from some regions are classified worse.

Other checks

07 · Classical NLP or an LLM?

Same task, same test projects, every kind of model

Field classification and climate / AI tagging on one random sample of projects starting in 2025 or later. LLMs answer in a validated JSON schema. An invalid answer falls back to the classical model and counts against the LLM's valid-output rate. The RAG variant shows the LLM the titles of the eight most similar training projects with their official labels. The wiki variants give it the two closest wiki pages, taken from the snapshot after the 2024 ingest so no page has seen a test project, optionally with each topic's official field profile computed in code.

08 · An LLM that keeps a wiki

Let the model maintain knowledge instead of re-reading abstracts

Classification is a poor job for an LLM; keeping a knowledge base current is a good one. The model replayed the portfolio one year at a time, 2022 to 2026, and kept one page per research topic up to date: themes, who does the work, how the topic evolved, links to related topics and the projects behind each claim. Every number comes from facts computed in code. A lint pass then checks every number, link and citation against the data before anyone reads the page.

    Lint

    Run by code after the last ingest. Pages with problems got one repair pass from the LLM, then were linted again.

    Editor review

    One LLM pass over the whole wiki looking for contradictions, overlapping pages and missing links. These go to a human curator; nothing is changed automatically.

    Same questions, answered from the wiki and with RAG

    From the wiki

    RAG over raw abstracts

    09 · How it is built

    Pipeline

    Ingest

    CORDIS bulk export

    Robust parser for CORDIS CSVs with inconsistent quote escaping. Projects joined with organisations, EuroSciVoc fields and policy markers in pandas.

    Text

    Tokenise & vectorise

    Accent folding, domain stop-words, uni- and bigrams, sublinear TF-IDF. spaCy POS tags and noun chunks for term extraction.

    Models

    Classify & embed

    One-vs-rest logistic regression with class balancing, hashing and LSA baselines, 96-dim LSA space, k-means topics, t-SNE map.

    Evaluate

    Temporal & fair

    Time-based split, threshold tuning on a held-out year, paired bootstrap, human-validated gold subset, per-region and per-field audit.

    Serve

    Browser inference

    Weights quantised to int8 and scored in plain JavaScript. No server, no data leaves the page. The same artefacts can sit behind a model server.