DEV Community

Jorge
Jorge

Posted on • Originally published at jorge.aguilera.soy on

Checking if some article in Dev.to is spam

These last days I’ve reported to the Dev.to admin a few posts as spam, so I’ve developed this small bash script to detect posible posts

INFO: You need to have httpie and jq installed. Also, an API-KEY is required

INFO: In fact, it’s more to practice httpie and jq filtering capabilities than a useful tool

#!/bin/bash
API_KEY=$1

(1)
latest=$(http https://dev.to/api/articles/latest api-key:$API_KEY accept:application/vnd.forem.api-v1+json per_page==80)

(2)
filtered=$(jq '.[] | select(.reading_time_minutes==1 and .user.user_id > 4)' <<< "$latest")

echo Total Last articles $(jq -M -r '.id' <<< "$filtered" | wc -l)
echo '-----'

echo Number of authors $(jq -M -r '.user.user_id' <<< "$filtered" | uniq | wc -l)
echo '-----'

(3)
users=$(jq -M -r '.user | .user_id' <<< "$filtered" | uniq)

for user_id in $(echo "$users"); do

   (4)
   strjoined_at=$(http GET "https://dev.to/api/users/$user_id" api-key:$API_KEY accept:application/vnd.forem.api-v1+json | jq -r '.joined_at')

   joined_at=$(date --date="$strjoined_at" "+%Y-%m-%d")
   days=$((($(date +%s) - $(date -d "$joined_at" +%s))/86400))

   (5)
   if (( ${days:-2} < 3 )); then
        echo "The $user_id user is suspect to be spam, see post:"
        jq --arg jq_user_id ${user_id} '.[] | select(.user.user_id == ($jq_user_id|tonumber)) | .url' <<< "$latest"
   fi
done
Enter fullscreen mode Exit fullscreen mode

| 1 | retrieve last articles (80 max) |
| 2 | filter by reading_time_minutes as spam usually are short post |
| 3 | extract uniques user_id |
| 4 | find user details for user_id |
| 5 | check if this account was recently created |

Obviously not all articles that meet these conditions are spam. Lot of people (as me) write a hello-world just created the account so the script show the url, so I can read the post and decide if it’s spam or not.

For next version, I have time, I would like to include some kind of "IA" to automatically read the post and decide if the post is spam

Top comments (0)