بسم الله الرحمن الرحيم
ورشة عمل وزارة الرياضة
جَمع وتَقييم وتَحليل المعلومات من المصادر المفتوحة
Concepts
Base Theory – Disinformation – Geopolitics
Saudi Society Model Results
SAVE TIME collecting data to focus on ANALYSIS
Updated: 12/08/2024
Course Goal: SAVE TIME collecting data to focus on ANALYSIS
- Prepare, Practice & Ethics
- OS: Windows
- Use Latest Chrome
- All tools are linked to virustotal.com – Please check before downloading
- Get results by being patient
- Practice Simulator with you own Scenarios
- The Ethics of Online Research:
- It’s ethical to harvest Public information without consent
- It’s NOT ethical to harvest Private information without consent
- For updates and help: madina[at]live.com
- WhatsApp: +966 555 311 961
—————————————-Examples—————————————
-
- PDF Links:
- Files:
- Analyzing Twitter
- MBC.zip – 317 MB – check with virustotal.com
—————————————-Homework-————————————–
—————————————-Online Tools————————————-
-
- YouTube:
- YT Comments Downloader to CSV
- YouTube Metadata Viewer – (offsite)
- YouTube Comment Viewer – (offsite)
- Snapchat
- TikTok
- Anonymous Viewers: urlebird.com or xaller.com – (offsite)
- Export TikTok video comments to CSV – Chrome Extension
- TikTok Downloader
- TikTok Monitoring
- Instagram
- Maps: South_Sana’a – Ayta_ash_Shab – Rafah
- EntiGram – Chrome Extension
- Anonymous Viewers: Picuki.com or Imginn.com
- Twitter:
- Deleted-Tweets
- Find-Twitter-id
- Export-Tweets (1) – Chrome Extension
- Export-Tweets (2) – Chrome Extension
- Telegram
- Image:
- Search by Image – Chrome Extension
- Image Forensic: (offsite)
- MediaInfo
- Image Verification – (Recommended)
- 29a.ch
- Tools to detect AI-generated images:
- Tools to aid in discovering images metadata and AI-generated images:
- Metadata viewers: ExifTool and Exif Pilot
- Image Zoom Tools:
- Search Pages:
- Company Vetting Summarizer
- Examples to search:
-
Churchill Ripley Group
-
DAZN Boxing LGBTQ+
- ASICS Japan
- Juul Labs LGBTQ+
-
Bahraini Sports Authority
- Nestle backlash, corruption and fraud
-
- Examples to search:
- OSINT SARAB – Concept
- OSINT SARABext
- Usernames
- Videos
- SearchEngines
- Maps
- LiveNews
- IP
- Documents
- Google Name Tool
- Company Vetting Summarizer
- Maps
- Realtime
- Marine Traffic
- Flight Traffic
- Cars Traffic
- Active Fire – Every 15 minutes – example: mapsfire.pdf / mapsfire.pptx
- EarthCam
- Bookmarklets
- Text Analysis: Voyant-tools.org
- Social Network Visualizer:
- YouTube:
—————————————-Software Tools———————————–
-
- TikTok
- Anonymous Viewers: urlebird.com or xaller.com
- TikTok Comments – virustotal.com
- Snapchat
- SnapChat_PROFILE.zip – check with virustotal.com
- SnapChat_GEO.zip – check with virustotal.com
- Use:
- Combine V – check with virustotal.com – to combine Videos
- Transcript – to transcribe video and/or audio
- General
- CheckUsername.zip – check with virustotal.com
- BackURL.zip – check with virustotal.com
- leaklooker.zip – How-to check with virustotal.com
- scrape.zip – check with virustotal.com
- Combine V – check with virustotal.com
- Transcript
- wget – Page
- youtube-dl & ffmpeg – Page
- Socialblade
- TikTok
—————————————-Resources-—————————————
-
- Sports Resources
- Encouraging Moderation – To Understand Extremism
- OSINT Framework – Updated from old site: osintframework.com
- datareportal.com – Digital Trending Reports
- World Blogs –Sorted by country and region
- statista.com – Statistics
- open.data.gov.sa – Saudi Datasets (2024)
- mandumah.com – Arabic Databases
- social-searcher.com – Search Social Media
- distill.io – Tracking website updates
- aleph.occrp.org – Investigative Reporting
- gijn.org – Investigative Reporting
- bellingcat.com – OSINT Investigative Reporting
- Political-stability-indexes
- perplexity.ai – AI generated reports
- boomlive.in – Misinformation on news and Social Media
- factcheck.afp.com – Misinformation on news and Social Media
- fatabyyano.net – Misinformation on news and Social Media
- TEXT Analysis:
- Voyant-tools.org – Text Analysis
- VoyantServer2_6_11.zip – To run Local
- Voyant-tools.org – Text Analysis
- Crowd Analysis
- dorksearch.com – Goggle advanced search operators
- google-hacking-database – Goggle advanced search operators
- btdig.com – BitTorrent
- Redddit Social – Sentiment Visualization (en)
- twitter.com/hashtag/OSINT – OSINT Hashtag in Twitter
- agda.ae – The Arabian Gulf Digital Archive
- skeptic.com – Scientific Skepticism
- offshoreleaks.icij.org
- Offshore Leaks (2013)
- Panama Papers (2016)
- Bahamas Leaks (2016)
- Paradise Papers (2017)
- Pandora Papers (2021)
- ipinfo.io – IP geolocation
- Videoindexer.ai
- skype2email – Skype to Email
- datasetsearch.research.google.com – Datasets Search
- visualdata.io/discovery – Images Datasets
- storage.googleapis.com/openimages – Images Datasets
- generated.photos/datasets – Images Datasets
- paperswithcode.com/datasets – ML papers, code, and datasets
——————————————–General—————————————
Social Media Limitations for Research
-
-
-
-
Twitter does not index all Tweets for search -
Twitter API fetch only last week - but not evenly! -
In Social Media “It seems to be pretty clear [from our study] that false information outperforms true information”
-
“People on Twitter like passing on lies better than they like retweeting the truth ... And bots aren’t necessarily to blame”
-
“Study: On Twitter, false news travels faster than true stories” -
Fake News in an Era of Social Media: Tracking Viral Contagion Is Social media Contagious? (1), (2), (3)
-
-
-
-
- Datasets
- SA Twitter Trending Hashtags – 2019-04-08 to 2022-08-012022-08-01
- Info guide
- How-to
- ta3lem Twitter Analysis – check with virustotal.com
- Sentiment analysis news datasets:
—————————————-Depreciated————————————-
-
-
- X (USE Twitter API)
- TwiProfile.zip – check with virustotal.com
- Download Excel Reporter for TwiProfile – virustotal.com
- TwiMultiUsr.zip – check with virustotal.com
- TrackTweet.zip – pdf – check with virustotal.com
- ScrapeTimeline.zip – check with virustotal.com
- extractTHash.zip – check with virustotal.com
- TweetHops.zip – txt – check with virustotal.com
- FakeFollowers.zip – check with virustotal.com
- TwiUserAnalysis.zip – pdf – check with virustotal.com
- TwiFolActivity.zip – pdf – check with virustotal.com
- TwiRetActivity.zip – pdf – check with virustotal.com
- Twitter_Sentiment.zip – (EN) – check with virustotal.com
- TwiUserG.zip – pdf – check with virustotal.com
- For drawing your networks, download and install:
- Custom Twitter Analysis examples:
- Compare_Hashtags.zip – pdf – check with virustotal.com
- #saudiarabia / #KSAatExpo2020 / #saudi / #KSA
- Range: 10-15/01/2022
- TwitterNews.zip – check with virustotal.com
- Search Keywords in: bbcnews, bbcbreaking, BBCPolitics, bbcworld, bbcengland, FinancialTimes, FT, guardiannews, Telegraph, TheSun, independent, france24, newsweek, ajenglish, AJENews
- Compare_Hashtags.zip – pdf – check with virustotal.com
- TwiProfile.zip – check with virustotal.com
- VersTube.zip – check with virustotal.com
- EMOJIS_Analysis.zip – check with virustotal.com
- TubeComments.zip – (AR) – pdf – Mdb – check with virustotal.com
- SaudiVision2030 Sentiments (en)
- Video Table – Page
- instagram-locations.zip – check with virustotal.com
- insFollowers.zip – check with virustotal.com
- insFriends.zip – check with virustotal.com
- insGetHashtag.zip – check with virustotal.com
- insTop10.zip – check with virustotal.com
- insTop20.zip – check with virustotal.com
- insUsrHashtag.zip – check with virustotal.com
- X (USE Twitter API)
-
والله ولي التوفيق