GitXplorerGitXplorer
v

Twitter-HashTag-Streamer

public
8 stars
3 forks
0 issues

Commits

List of commits on branch master.
Verified
603713a82f9e2a241a211c3f9136cbfd6b0b26d6

Merge pull request #3 from varunu28/dependabot/pip/requests-2.20.0

vvarunu28 committed 3 years ago
Verified
dea816994ad22126140d4c30d9f0bb60a6906186

Bump requests from 2.19.1 to 2.20.0

ddependabot[bot] committed 3 years ago
Verified
6f1268596333938ab82ec836fdd0c68d4a1d54f6

Merge pull request #2 from varunu28/dependabot/pip/pyyaml-5.4

vvarunu28 committed 3 years ago
Verified
9865b07cec88774aa7bf3eaaede57d83f124d51b

Merge pull request #1 from varunu28/dependabot/pip/urllib3-1.26.5

vvarunu28 committed 3 years ago
Verified
96dfa98069860f82e8fc82dccc39bd38258597cf

Bump pyyaml from 3.13 to 5.4

ddependabot[bot] committed 3 years ago
Verified
d92cc047dbd2e23d6e19411b68894ca72c4218ed

Bump urllib3 from 1.23 to 1.26.5

ddependabot[bot] committed 3 years ago

README

The README file for this repository.

Twitter-HashTag-Streamer

A Python script to stream live tweets related to a list of Hashtags and save it to an S3 bucket as a JSON file

Demo

To-do

  • [X] Get Twitter API key
  • [X] Create streaming application to stream live tweets
  • [X] Create S3 config using boto framework
  • [X] Save every new tweet as a JSON to the S3 bucket
  • [ ] Add multithreading for large streamings

How to run the script?

  • Install the required libraries by running pip install -r requirements.txt in a virtual environment.
  • Create a Twitter application on the Twitter developer portal
  • Generate the token and and key for the application
  • Create a file credentials.py and fill the key and tokens in below template
    ACCESS_TOKEN = ""
    ACCESS_TOKEN_SECRET = ""
    CONSUMER_KEY = ""
    CONSUMER_SECRET = ""
    
  • Make sure to have s3 permissions for the aws user which has been configured on your system
  • Update the hashtags and bucket name in config.yaml
  • Run the script