Public tools, prototypes, and implementation patterns built or adapted for The Unjournal
Academic and applied-stream evaluation forms with guided metrics, credible intervals, calibration practice, and export formats.
Systematically highlight and categorize claims in research papers using GiveWell-style assessment methodology. Export as HTML, CSV, or JSON.
Compare human expert critiques with LLM-generated issue assessments. Score matches and export annotations for analysis.
Suggested AI and analysis tools for evaluators, including RegCheck, RoastMyPost, NotebookLM, Elicit, and COS TOP Guidelines. Includes Unjournal AI policy.
Exploratory dashboard and overview for high-impact legal scholarship, including AI-assisted candidate triage and a public planning page for possible evaluation pilots.
Searchable database of explicit disagreements, uncertainties, and "what would change my mind" statements from EA Forum and LessWrong posts, to inform research prioritization.
Guides for using AI tools (NotebookLM, ChatGPT Pro, Elicit) to assist with research evaluation while maintaining critical judgment.
Training materials for evaluation workshops, including exercises, calibration games, and instructional content.
Internal meeting availability collection, shared response views, and best-time ranking for field specialist, management, advisory, and related meetings.
Examples of tools we have built or adapted. Some are organization-specific or still evolving, so this section describes the pattern without linking internal systems or presenting them as polished products.
Collect registrations, availability, pre-reading responses, and structured forecasts or beliefs; then bring the results into a shared workshop record and follow-up materials.
Turn workshop notes, transcripts, and written contributions into an editable synthesis of points of agreement, disagreements, open questions, and next steps. AI can prepare a first pass; a responsible person checks and revises it.
Gather candidate research from multiple sources, enrich basic metadata, flag likely relevance, and keep a human-reviewed queue rather than relying on an unstructured inbox or ad hoc list.
Move a structured expert assessment through validation, editing, and a publication-ready draft, while keeping the final editorial and publication decisions with people.
Periodically check selected public sources for new citations, paper versions, or feedback; put only the potentially useful changes into a review queue with source links and a short summary.
Collect selected public posts and discussions, flag items that may merit a response or share, and prepare a short, checkable summary for a person to decide whether and how to engage.
Collect receipts and expense details, extract the basic fields, and assemble a checked queue for reimbursement or accounting. People review the evidence and approve the final records.
OpenAlex-based citation network analysis for tracking papers that cite Unjournal-evaluated works and identifying research influence patterns.
Analyze whether authors updated their papers after receiving Unjournal evaluations. Compares before/after versions and evaluator suggestions.
Python scripts for automated issue matching using sentence embeddings and GPT API. Generates data for the Issue Annotation UI.
Continuously updated public outputs rather than standalone tools.
Interactive dashboard of The Unjournal's completed evaluations and ratings — explore evaluator scores, credible intervals, and summary metrics across all papers.
Hub for The Unjournal's Pivotal Questions workshops (wellbeing, cultured meat, plant-based substitution), including scheduling and beliefs-elicitation forms.
Adjacent projects from The Unjournal's orbit — related, but not core to the evaluation mission.
Searchable directory of organizations and products with documented, verified impact claims — a related spinoff project, not a core Unjournal evaluation tool.