LOL Bench
LOL Bench is an open-source benchmark that measures whether 13 large language models can explain jokes, write original jokes, and distinguish funny jokes from bad ones. Each model receives a composite humor score with confidence intervals, and the benchmark pairs every score with estimated inference cost per call. The tool breaks down model performance by six joke mechanism categories to expose which models struggle with specific comedic structures like wordplay or absurdist humor. Results are published on a public leaderboard and raw data is available on GitHub. Agencies use LOL Bench to compare model humor performance and cost efficiency before selecting an LLM for creative copywriting work.
LOL Bench is an open-source benchmark, priced at $1/month on the Score against cost plan. InnovaAI scores it 4.1/10 for agency adoption, best for Creative Director, Copywriter, and Founder roles handling weekly client-facing work.
Agency Audit
LOL Bench is an open-source benchmark that scores 13 large language models on humor comprehension, joke writing, and joke ranking, paired with per-model inference costs. Agencies selecting LLMs for creative copywriting workflows benefit most: your team can compare model humor performance against cost efficiency before committing to a production API. The benchmark breaks down failure modes by six joke mechanism categories, exposing which models struggle with specific comedic structures. Best for creative directors, copywriters, and founders evaluating whether a cheaper model can replace a pricier one without sacrificing humor quality in client deliverables.
3recommended
24/mo
$1,799/mo
Low
Illustrative scenario. Not a guarantee. Net capacity is the value of reclaimed time at $75/hr, less the lowest verified paid base plan (flat plan cost is shared). Hours saved come from the service estimate; implementation, taxes, and unprovided usage charges are excluded.
- Creative Director handling LLM model selection for creative copywriting
- Copywriter handling humor quality benchmarking before API contract commitment
- Founder handling cost-efficiency analysis for LLM provider negotiation
- Your agency does not produce comedic or humorous copy for clients, or humor represents less than 5% of your LLM usage. LOL Bench benchmarks only humor tasks and will not inform model selection for strategy, technical writing, or serious copywriting workflows.
- Your team has already standardized on a single LLM provider and does not plan to evaluate alternatives. LOL Bench is a comparison tool; it adds no value if you are not actively choosing between models.
- Your creative team works with proprietary or fine-tuned LLM models that are not included in the 13-model leaderboard. LOL Bench does not support custom model benchmarking, so you cannot test your internal or client-specific models.
Internal Adoption Path
$1/mo
$1/mo flat plan
24 hr/mo
3 seats × 8 hr each
$1,800/mo
modeled at $75/hr labor rate
$1,799/mo
value − subscription cost
In this model, 3 seats reclaim 24 hours of team time each month. Valued at $75/hr that is $1,800/mo, and after the $1/mo subscription it leaves $1,799/mo of capacity for billable client work.
Illustrative scenario. Not a guarantee. Uses the lowest verified paid base plan. Implementation, taxes, and unprovided usage charges are excluded.
Platform Features
Core capabilities of LOL Bench
Humor comprehension scoring
Scores each model on its ability to explain why a joke works by comparing model explanations against human-written joke notes. Copywriters and creative directors use this score to identify which models understand comedic structure before deploying them on client work.
Blind pairwise joke-writing contests
Two models generate jokes on the same premise; human voters pick the funnier line without knowing which model wrote it. Your team uses voting results to validate benchmark rankings and catch models that score high on explanation but fail at original humor generation.
Cost-efficiency comparison matrix
Pairs each model's humor score with estimated per-call inference cost, letting your operations or finance lead calculate whether switching to a cheaper model saves money without sacrificing joke quality. Founders use this to justify LLM contract negotiations.
Failure-mode breakdown by joke mechanism
Breaks down model performance across six joke categories (F1 through F6), exposing which models struggle with specific comedic structures like wordplay, timing, or absurdist humor. Creative directors use this to route different joke types to different models or avoid models that consistently fail on your client's preferred humor style.
Crowdsourced funniness ranking
Collects human votes on 1,500 model-written jokes to build a ranking of which jokes humans find funniest. Your team can review top-ranked jokes to see real examples of what each model produces at its best.
Open benchmark data and GitHub repository
Publishes raw results, confidence intervals, and full dataset on GitHub. Your team can audit methodology, download data for internal analysis, or integrate benchmark results into your model selection documentation.
What Makes LOL Bench Different
Unique advantages vs similar tools in this niche
Scores humor comprehension against human-written joke notes rather than model self-judgment
vs Generic LLM leaderboards that rely on model-as-judge scoringTwo models from other labs grade each answer, and no model ever rates a punchline.
Pairs every benchmark score with the estimated dollar cost of running the full set
vs Leaderboards that report quality without cost contextglm-5.3-flash scores 97.3 for an estimated $0.04 while muse-spark-1.2 reaches 95.3 at $0.01.
Isolates humor failure modes by six joke mechanism categories
vs Single aggregate humor scores that hide where models break downEvery model scores lowest on F6, from mimo-v2.5 at 81 to qwen3.8-max at 92.
Value Equation
Outcome-likelihood-time-effort assessment for LOL Bench
Limited agency channel
LOL Bench scored below the agency-resellability threshold (agency_fit_score < 50). The Value Equation projects agency-side outcomes, which don't apply to tools without a clear resell pathway.
Contact LOL BenchPricing
LOL Bench platform cost to your agency
Score against cost: $1/mo
Score against cost
- dots are models · the orange bar is the range · lime = best score on the board
- glm-5.3-flash97.3±0.
No verified white-label program for LOL Bench: client-facing delivery runs under the platform's native branding.
Market Intelligence
Offer + scale economics for LOL Bench
Limited agency channel
LOL Bench scored below the agency-resellability threshold (agency_fit_score < 50). It's a useful tool but not designed for white-labeled or retainer-based reselling, so we don't publish productized offer economics for it.
Contact LOL BenchInvestment Decision Framework
Strategic vetting analysis for LOL Bench
Situational Fit
Fit depends on your client mix
Buy If
4Your creative team writes comedic copy for clients 5+ hours per week and currently relies on trial-and-error testing across multiple LLM APIs to find the right model for humor tasks. LOL Bench eliminates weeks of manual A/B testing by publishing ranked humor scores with confidence intervals upfront.
Your copywriters or creative directors spend time and budget testing different LLM providers to see which one generates funnier headlines, taglines, or social media jokes. LOL Bench's pairwise voting system and per-model cost breakdown let you pick a model before spinning up paid API calls.
Your team evaluates LLM cost efficiency and wants to know whether a cheaper model like muse-spark-1.2 (95.3 humor score) can replace a premium option like claude-opus-5 (96.5 score) without losing quality. The benchmark pairs each score with estimated inference cost, so you can calculate payback period on model switching.
Your founder or operations lead is building an LLM selection rubric for creative work and needs objective, third-party data on model humor performance to justify API contract decisions to stakeholders. LOL Bench publishes raw data and confidence intervals on GitHub, giving you auditable evidence.
Skip If
4Your agency does not produce comedic or humorous copy for clients, or humor represents less than 5% of your LLM usage. LOL Bench benchmarks only humor tasks and will not inform model selection for strategy, technical writing, or serious copywriting workflows.
Your team has already standardized on a single LLM provider and does not plan to evaluate alternatives. LOL Bench is a comparison tool; it adds no value if you are not actively choosing between models.
Your creative team works with proprietary or fine-tuned LLM models that are not included in the 13-model leaderboard. LOL Bench does not support custom model benchmarking, so you cannot test your internal or client-specific models.
Your workflow requires real-time model performance feedback integrated into your copywriting tools or CMS. LOL Bench publishes static benchmark results; it does not offer API access to live humor scores or automated model routing.
Bottom Line
LOL Bench is an open-source benchmark that scores 13 large language models on humor comprehension, joke writing, and joke ranking, paired with per-model inference costs. Agencies selecting LLMs for creative copywriting workflows benefit most: your team can compare model humor performance against cost efficiency before committing to a production API. The benchmark breaks down failure modes by six joke mechanism categories, exposing which models struggle with specific comedic structures. Best for creative directors, copywriters, and founders evaluating whether a cheaper model can replace a pricier one without sacrificing humor quality in client deliverables.
Reality Check
LOL Bench is a reference tool, not a production integration. Your team must manually review benchmark results and make model selection decisions; there is no API that auto-routes requests to the best-performing model. The benchmark covers only humor tasks, so it does not inform model choice for non-comedic copywriting, strategy, or technical work.
Low effort: self-service setup with guided onboarding
Academy for LOL Bench
Work through it in order: the course for this service first, then the modules behind it.
No Academy modules are published for this service yet. Browse the full Academy
Why this category matters
The commercial case before the tooling.
Core concepts
The mental model you need to price and scope the work.
- Evaluation Debt RatioConcept
Evaluation Debt Ratio measures the gap between how much testing an AI system receives and how much it needs given its production stakes. Agencies often ship client AI features with only ad-hoc checks, treating evaluation as a post-launch afterthought. This framework forces a deliberate calculation: for every dollar of client retainer or every hour of agent runtime, how much evaluation coverage exists? A low ratio means high risk of unpredictable outputs, hidden cost spikes, and reputational damage. For example, a client-facing chatbot handling refunds needs rigorous evaluation, while an internal summarization tool can tolerate lighter checks. Tools like Langfuse, Braintrust, and Arize provide tracing and scoring to quantify this debt, but the framework applies even without them: track the number of test cases per production interaction. Agencies that close the evaluation debt early can charge a premium for 'production-ready' AI, while those that ignore it face client churn.
- Observability-Led Pricing PremiumConcept
Agencies that embed evaluation and observability infrastructure into their AI delivery can charge a premium for 'production-ready' AI, while those that skip it face unpredictable failures and client churn. This framework argues that the depth of observability a client can see directly correlates with the price they will accept. For example, an agency using Langfuse to trace every agent call and surface cost, latency, and quality metrics can present a transparent dashboard that justifies a higher retainer. Conversely, a client whose AI misbehaves with no traceability will demand discounts or leave. The 2026 n8n analysis warns that self-reviewing LLM loops compound errors, making observability a non-negotiable for trust. Agencies that instrument early de-risk deployments and convert transparency into margin.
- Trace-to-Test Feedback LoopConcept
The Trace-to-Test Feedback Loop is a framework for turning production observability data into a continuously improving evaluation suite. Instead of relying on static test sets, agencies capture real user interactions from tracing tools, identify failures or edge cases, and convert them into regression tests. This loop tightens the gap between what happens in production and what is tested pre-deployment. For agencies, this means fewer surprise failures on client deployments and a defensible story for 'production-ready' AI. For example, a platform like Langfuse provides hierarchical traces of every LLM call, which can be mined for problematic patterns. Those patterns become new evaluation cases in a tool like Braintrust, where teams define scoring criteria and run them at scale. The result is a living evaluation pipeline that improves with every client interaction, reducing drift and building client trust.
Decision and risk
How to judge the fit, and the ways it goes wrong.
- AI Evaluation Rule: Trace Before You TrustEvaluation Rule
Deploy tracing and evaluation pipelines before any client-facing AI goes live, and treat observability as a billable deliverable.
- When Agent Outputs Feed Client Workflows, Gate Them With EvalsEvaluation Rule
Deploy an evaluation and observability layer before any agent output reaches a client deliverable.
- The Dashboard-Only Trap in AI Evaluation & ObservabilityFailure Pattern
- The Evals-Before-Observability Trap in AI Evaluation & ObservabilityFailure Pattern
8 modules selected for LOL Bench
Frequently Asked Questions
Answers about pricing, setup
LOL Bench offers 1 pricing tier, at $1/mo (Score against cost).
LOL Bench is open-source and free to use. The benchmark results and leaderboard are published on the website and GitHub at no cost. There is no per-seat pricing or subscription fee for accessing benchmark data.
Creative directors and copywriters use LOL Bench to evaluate which LLM produces the funniest copy for client campaigns before committing budget to API calls. Founders and operations leads use the cost-efficiency matrix to negotiate LLM contracts and justify model selection decisions. Project managers reference the benchmark when scoping creative work to set realistic expectations for humor quality across different models.
If your copywriting team currently spends 3 to 5 hours per week testing multiple LLM APIs to find the best model for humor tasks, LOL Bench eliminates that testing cycle by publishing ranked results upfront. Conservative estimate: 2 to 4 hours per week per copywriter, assuming your team adopts the benchmark results instead of running parallel API tests. Payback is immediate if you switch to a cheaper model without losing humor quality.
No. LOL Bench benchmarks only the 13 published models on the leaderboard. The tool does not support custom model testing or proprietary LLM evaluation. If your team uses internal or client-specific fine-tuned models, you would need to run your own humor benchmark separately.
No. LOL Bench is a reference benchmark, not a production integration. Your team reviews the published leaderboard and manually selects a model based on humor score and cost. There is no API, plugin, or direct integration with copywriting software, design tools, or content management systems.
LOL Bench publishes results in waves. The current version (v0.2.0) includes 13 models scored on humor comprehension and joke writing, with joke ranking data still being collected. The benchmark is open-source on GitHub, so your team can track updates and new model additions as they are released.
LOL Bench is a public benchmark with no user accounts or data storage. Your team accesses published results on the website or GitHub; there is no personal data, project files, or usage history tied to your agency. If you stop visiting the site, nothing is deleted or lost.