# Source: https://docs.avaamo.com/user-guide/llamb/llamb-filters/social-filters.md

# Social filters

In simple terms, `Social filtering` is the process of identifying and managing content that is inappropriate, harmful, or irrelevant based on certain criteria in any of the following categories - Racism, Hate, Violence, Self-harm, Sexual harassment, Discrimination, and Drugs.

### Social filter categories <a href="#kaurp122tf8c" id="kaurp122tf8c"></a>

Using the neural multi-class classification models, the `Social filtering` system in LLaMB identifies and determines its severity based on predefined criteria in both input prompts and output completions in the following main categories:

<table><thead><tr><th width="181">Category</th><th>Description</th></tr></thead><tbody><tr><td>Racism</td><td>Content promoting discrimination and prejudice against people based on their race or ethnicity.</td></tr><tr><td>Hate</td><td>Content promoting hate against specific groups.</td></tr><tr><td>Violence</td><td>Content promoting violence against specific groups, endorsing illegal weapon use, or providing related instructions, or encouraging or aiding in various criminal activities</td></tr><tr><td>Self Harm</td><td>Content encouraging self-harm or lacking appropriate health resources.</td></tr><tr><td>Sexual harassment</td><td>Encouraging sexual acts, particularly with minors, or explicit content.</td></tr><tr><td>Discrimination</td><td>Content promoting unfair or prejudicial distinctions between people based on the groups, classes, or other categories to which they belong or are perceived to belong, such as race, gender, age, religion, physical attractiveness, or sexual orientation.</td></tr><tr><td>Drugs</td><td>Promoting illegal production or use of controlled substances.</td></tr></tbody></table>

### How does it work? <a href="#jchdzyg4ucso" id="jchdzyg4ucso"></a>

The following are some of the key features that describe how social filtering in LLaMB works:

* **Real-time evaluation:** `Social filtering` is applied synchronously, meaning that both the input prompts and the generated outputs are evaluated in real-time. This ensures that the harmful content is identified and filtered before it can be displayed or processed further.
* **Zero tolerance against harmful content:** The default social filtering configuration is set to filter at the **strict** severity threshold for all the [listed content harm categories](#kaurp122tf8c) for both prompts and completions, implying a zero-tolerance approach for both input prompts and output completions.
* **Customized blocklist for output content**: Blocklists are used to prevent specific terms or patterns from appearing in the output generated by LLaMB. These blocklists can be customized as per your business requirement and aim at ensuring the responsible use of LLaMB by blocking inappropriate, harmful, or undesirable content. When a blocklist is applied to a social filter, any prompt or generated content that matches an item in the blocklist is blocked.
* **Prompt shields against jailbreak attacks, and indirect attacks**: These shields are designed to filter or annotate user prompts that might present a risk of bypassing social filters. LLaMB provides a framework to define and enforce rules around acceptable topics, responses, and dialogue paths, it prevents harmful or inappropriate outputs from large language models. This is particularly important for protecting against vulnerabilities like jailbreaks and prompt injections. Rules are used to:
  * Either reject the input, stop further processing, or adjust the input to ensure safety and appropriateness.
  * Either block the output, preventing it from reaching the user, or modify it to ensure it aligns with content guidelines and user expectations.
  * To handle messages in their standard forms and determine whether to act, call the LLM for the next step or a response, or choose a predefined reply, ensuring the conversation stays relevant and appropriate.
  * Reject a chunk, stopping it from being used to prompt the LLM, or modify the relevant chunks to ensure that only accurate and safe information is utilized.
* **Policies to safeguard content**: LLaMB defines policies that help in safeguarding both inputs (prompt classification) and response classification. It helps in indicating whether a given prompt or response is safe/unsafe, and if unsafe based on a policy.

### Examples <a href="#k64k4zgshf79" id="k64k4zgshf79"></a>

**Example 1**: The following example illustrates an example LLaMB agent and how the **Zero-tolerance** threshold filters the content from generating derogatory responses:

<div align="left"><img src="https://2934665269-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LpXFTiTgns4Ml77XGi3%2Fuploads%2FkkTc0npAx64TLfBkEhVF%2F0.png?alt=media" alt="" width="375"></div>

**Example 2**: A few other instances to illustrate how LLaMB social filters detect inappropriate input prompts and prevent generating any harmful responses with its Strict threshold policy:

<div align="left"><img src="https://2934665269-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LpXFTiTgns4Ml77XGi3%2Fuploads%2FTYLJTpZw3Sjx7nVof34J%2F1.png?alt=media" alt="" width="375"></div>

**Example 3**: The following example demonstrates how prompt shield works against jailbreak attacks:

<div align="left"><img src="https://2934665269-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LpXFTiTgns4Ml77XGi3%2Fuploads%2FOUCWNUzIbNQPac3lieb0%2F2.png?alt=media" alt="" width="375"></div>