TrustGuard AI

A 6-layer multi-modal defense system and trust-based recommendation pipeline protecting revenue from bot farm manipulation.

Open repository

Project claim

6-layer multi-modal defense against bot farm manipulation.

Role

Lead Big Data Engineer

Focus signals

1.98M reviews25% RMSE gainbot farm clustering

Proof signal 1

25% RMSE improvement

Proof signal 2

Processed 1.98M reviews

Proof signal 3

6-layer user defense engine

Challenge

Fake reviews pollute recommender models, causing poor suggestions and revenue losses.

Solution

Built a Lakehouse architecture pipeline filtering reviews with a multi-layered behavioral defense, training ALS models only on gold-layer trusted reviews.

Build notes

Avoided OOM crashes by parallelizing feature extraction in Spark worker partitions

Cached dataframes (.cache) to speed up iterative K-Means clustering calculations

Substituted Pandas matrix operations with Spark MLlib VectorAssembler and HashingTF

Tools + stack

PySpark, Spark MLlib, K-Means, ALS, and Streamlit.

PySparkSpark MLlibStreamlitPythonVADERK-MeansALS

RMSE reduction

25% improvement

Protected revenue

$20M+ estimated

Data size

1.98M reviews

Architecture flow

Step 1

raw reviews

Step 2

pyspark features

Step 3

6-layer trust score

Step 4

trusted data gold layer

Step 5

als recommender

Step 6

streamlit interface

System Architecture

TrustGuard runs PySpark feature extraction, feeds values into a 6-layer user defense score, stores gold-layer reviews, and trains ALS recommending logic.

6-Layer Defense

Layers include User Clustering (K-Means), Sentiment-Rating Inconsistency (VADER), Text Similarity (Cosine), Burst Detection, Rating Entropy (Shannon), and the composite Trust Index.

Decision signals

Quality data beats volume: training on less but cleaner data yields superior models

User behavior (frequency, deviation) yields stronger fraud signals than text classification alone

Compute trust indexes dynamically to prevent downstream metric pollution

Outcomes

Filtered out 1.6M dirty reviews to achieve an RMSE of 0.84 vs 1.12 baseline

Developed a 6-layer defense system detecting sentiment-rating inconsistency, similarity spam, and burst temporal anomalies

Quantified business value by establishing a Revenue Protection Model preserving $20M+ from returns