Cookbooks showcasing various applications of Cleanlab, as well as code shared for: education, reproducibility, transparency.
| Example | Description |
|---|---|
| TLM-Demo-Notebook | Demo-ing various applications of the Trustworthy Language Model, particularly in customer support |
| tlm_call_api_directly | Call the TLM REST API directly. You can use any programming language (eg. Typescript) with http lib/tools by providing the necessary payload and headers. |
| TLM-PII-Detection | Find and mask PII with the Trustworthy Language Model |
| Detecting GDPR Violations with TLM | Analyze application logs using TLM to detect GDPR violations |
| Customer Support AI Agent with NeMo Guardrails | Reliable customer support AI Agent with Guardrails and trustworthiness scoring (Nvidia Blogpost) |
| Better LLM Evals in MLFlow | Automatically find the bad LLM responses lurking in your production logs/traces via trustworthiness scoring in MLFlow |
| TLM-Record-Matching | Using the Trustworthy Language Model to reliably match records between two different data tables |
| TLM-SimpleQA-Benchmark | Benchmarking TLM and OpenAI LLMs on the SimpleQA dataset |
| benchmarking_hallucination_metrics | Evaluate the performance of popular real-time hallucination detection methods on RAG benchmarks |
| benchmarking_hallucination_model | Evaluate the performance of popular hallucination detection models on RAG benchmarks |
| fine_tuning_data_curation | Use Cleanlab TLM and Cleanlab Studio to detect bad data in instruction tuning LLM datasets |
| few_shot_prompt_selection | Clean the pool of few-shot examples to improve prompt template for OpenAI LLM |
| fine_tuning_classification | Use Cleanlab Studio to improve the accuracy of fine-tuned LLMs for classification tasks |
| generate_llm_response | Generate LLM responses for customer service requests using Llama 2 and OpenAI's API |
| gpt4-rag-logprobs | Obtaining logprobs from a GPT-4 based RAG system |
| fine_tuning_mistral_beavertails | Analyze human annotated AI-safety-related labels (like toxicity) using Cleanlab Studio, and thus generate safer responses from LLMs |
| Evaluating_Toxicity_Datasets_Large_Language_Models | Analyze toxicity annotations in the Jigsaw dataset using Cleanlab Studio |
| time_series_automl | Model time series data in a tabular format and use AutoML with Cleanlab Studio to improve out-of-sample accuracy |