1. Extract the .app from dmg file to your project’s folder.

2. Right click the .app file and click show package contents.

3. There you’ll find a couple of files and folders. Copy the ‘Resources‘ folder to the parent project directory

   

4. We are gonna need the following tools :

   1. asar CLI : its basically a packaging program for electron apps which can be also be used for unpacking apps

   2. webcrack CLI : Really Good JS deobfuscator. Works well on common obfuscation techniques. I’ll show what they are later.

   3. A faster text editor : Obfuscated and packaged js code is reeeally ugly and the LOC can extend upto 100k+ per file. So no, your electron based editor (cough.. vs code) won’t cut it. You need something performant to perform static analysis without ripping your hair out.

      

      I recommend using vim or any other cli based IDE. You can also try ZED if you are used to the UI. It’s written in RUST and is really performant.

Step 2. Unpack the asar file : Use the asar CLI to unpack the asar file: Make sure the supporting directories (eg. app.asar.unpacked) and modules are present in the folder where app.asar is present.

npx asar extract resources/app.asar extracted-app/

This will extract out all the app data to a separate folder for static analysis.

Step 3. Find and deobfuscate all the {{required}} js files in the dist directories …

Obfuscated js looks like this :

And this is the result after deobfuscation :

Makes the code much more readable..

Obfuscation is a facinating topic overall. You can read more about it here.

Step 4. Static Analysis: Find the code that is responsible for your desired functionality.

I will take an example of bypassing licence activation.

Note: I already own the app I am demonstrating this for. I don’t condone piracy. Just fkin pay for good software.

This is the license activation screen.

Use search to find the exact phrase in the codebase..

Use this to find out what function is getting called when you press the activate button. then back-track to find out the function where network call is being made. That is made typically to a licensing server.

Other ways to find out the code responsible for activation is :

• Network monitor. Start the electron app connect to postman’s proxy and intercept all the request: this may or may not work because the developers may not use simple protocalls to make requests.

• Dev console. Use electron’s inbuilt dev-console to find out the request. This may also not work deu to the same reason above.

• Wireshark/Burpsuite. Can work, have’t tried yet.. Wireshark support a large variety of protocols so you may find out server endpoint using it.

• Search for https:// in the codebase. This is tricky and time consuming.

After you’ve found out what part of code is responsible for making the request. Modify it so that it always returns a success. Also check all the downstream functions of that piece of code to bypass all of those checks too..

In my case I found out this part of code :

It can simply be replaced by this to bypass all checks:

Another part was the the function which persists the licence key across sessions :

This can also be simple bypassed:

Similarly you can find out all the places licence key is being fetched/stored/verified and mod out the functions.

Step 5. Repackaging and assembling a new .app file.

Now after you’ve made all the changes. Repackage the app to an asar file using the asar cli again.

npx asar pack extracted-app/ app.asar

Now go to the original .app of your application, right click > show package contents > delete the original app.asar from the Resources folder and copy the modified newly packaged app.asar into it.

Reopen the app and see if your mods had any effect.

Note: Open the app without mods first just for macOS to pass package verification. Subsequent modifications will pass through just fine.

Reverse engineering apps, especially obfuscated ones is really fun. It’s like solving an interesting math problem.
Will post more about the interesting/challenging techniques I encounter while reverse-engineering random apps in the future.

Adios till then 🙌

The app also has a feature to turn of the electricity supply. Which is… interesting. I really wanted to reverse engineer and pentest the app for vulns. So one fine weekend I decided to do just that.

Found some major-major privacy gaps (not validating the bearer when fetching my neighbour’s meter details ::skull_emoji::) which can be another blogpost in itself once they patch it.

For now we’ll be focussing on the turning off the supply part.

I used burpsute, frida and bluestacks to intercept the API requests as I demonstrated in detail in a previous blog post.

But the major steps remain the same :

1. Setup burpsuite proxy.

2. Connect to the burp proxy using any vpn app.

3. Use xposed to install FRIDA framework onto device.

4. Install the burp certificate.

5. Install the target app and turn on intercept to see the intercepted requests.

This gave me the API endpoints that are called when the user hits the “turn meter off” button.

Now my idea was to use apple shortcuts app to simulate this request but it does’t allow me to put body data or auth tokens with the request. It just has a feature to call a URL with some parameter inside of it.

So to simulate this request I created a serverless function on deno deploy(tested out with cloudflare workers but it mysteriously refused to work on cloudflare).

After making two endpoints for the ON and OFF functionality, I created a new http shortcut on the iOS shortcuts app to trigger one of the endpoints whenever I leave or enter my apartment based on my geo location.

It works like a charm 80% of the time minus some random GPS fuckups and misfires. There seems to be some cool-down of about 5 minutes between on and off states to prevent abuse.

Note: The LLM landscape has changed quite a lot since September, when I built this, so all the discussion in this post might not be completely relevant.

One of the two products I built was RakuMend, a conversational shopping and recommendation engine powered by GPT3.5 and Google’s vertex AI for image captioning needs (GPT4 Vision wasn’t a thing at that time).

Rakuten has a global website, rakuten.com, which is basically a brand aggregator, and they get a cut every time a purchase is made through it.

I wanted to build a recommendation system around it, so it takes a natural language query (e.g., I am going to a work trip this weekend; suggest something comfy yet semi-formal) and gives out product recommendations from the inventory.

V1: ViT/CLIP + Cosine Similarity/Vector Search

The first version was powered by a simple vector search engine in which I scraped approximately 10,000 products from Rakuten Ichiba (their Japanese counterpart). Divided into two product categories, tops and bottoms. I used Clip (OpenAI’s image captioning model) to get image vectors and did a cosine similarity search to find matching outfits from the DB. I used pinecone for my vector DB, but really, a numpy dict would have worked just fine.

It worked «fine». It was too slow to be used on any production workload with 100s of categories. Also the recommendations weren’t that great tbh.

So onto the next version.

V2: GPT Functions and Full Text Search

Not having access to the whole product catalog was a bummer. But Rakuten’s full text search is really good.

So my idea was to leverage FTS with GPT function calling to get product ideas from GPT, generate search queries from it, scrape the top 5 results, and display them.

This was around the time when GPT unveiled function calling and models specifically tuned for that (instruction tuned).

A Note about GPT Function Calling: Its not intuitive at all and is quite unreliable. I don’t know anyone using function calling in production. I recommend writing your own parser (do a simple regex on return streamed responses and manually call the function in Python).

This is almost like retrieval-augmented generation (except we use full text search instead of vector similarity to find the top k results):

This is the final product I demoed at the hack.

How it works and the code:

This is the scraping module that searches and returns the top n products for the given query from Rakuten’s website :

This is the fastAPI endpoint that handles all the messages as well as user’s input images :

The high_def flag is a flag I implemented when I wanted to get the image captioning results from reverse-engineered Bard’s web API instead of Vertex’s PALM API.

The previous messages are separated by a % delimiter. Each query sends a user preference (age and gender) with it to give our LLM more context about the user.

This is the gpt responder module : It just parses the current user query and previous queries. Also appends system prompt before passing the whole thing to GPT3.5 api. LLM can be hot-swapped.

Important Note: The key to making sure your instruction-tuned models follow instructions in long context lengths is to append the system prompt again at the end of the dialog chain. This has to do with the LLM architecture and how the attention heads function. You might not have to do this at all, depending on the model (e.g., mistral models are pretty good at this). This thing wasted a whole lot of my time figuring it out at the hack.

Parse and replace module: regex(es) the GPT response and returns a structured output after searching for each term using rk_search module. This output is then rendered by my front end.

Image captioning : Added a feature that allowed users to add their own outfits and clothing items to chat and ask LLM to give recommendation about it.

Used google’s palm API (and bard’s reverse engineered API) to generate highly detailed image captions which were then fed to my LLM.

Here’s a demo of the feature :

Last but not least these were my prompts:

https://text.is/8Y0Y

Here’s another demo vid of the complete project:

This thing, coupled with another product I build (RakuTry, a virtual try-on extension [writeup soon]), was enough to get me a second (first being an internal team [usual tbh]).

GG’s

The only thing I miss is KDE Connect. More specifically, I miss the ability to receive my Android Push notification on my PC.

None of the ready-made solutions are simple or OSS enough, so I decided to build something myself.

Web-Push notification’s have been pretty standardized as of now, with even iOS supporting it. So I decided to build something on top of this.

The basic idea goes something like this:

The Android app will run in the background forever using NotificationListenerService and send all the notification parameters to my server via a post request. The server will then validate the request and do some basic filtering (spam checks and app validation). Post-filtering, it’ll send a web-push notification to all the devices I’ve subscribed to (mac,iOS, etc., doesn’t matter; basically anything with a browser in it).

I decided to do validation and filtering on the server instead of doing it on the device because (a) recompiling or installing the app for a new ruleset is a chore as opposed to just pushing the changes on the server and letting github actions do the magic, and (b) I can use much more powerful filtering on the server than on the device.

I haven’t done Android development or written Java in a long, long time, so I decided to use GPt4 to help me with that. I realized that it is terrible at writing anything but Python.

So a giga-ton of prompts and pasting Android API references in the context window later... It finally worked (sort of) in just about an afternoon of work.

Here’s some ugly code if you want to take a look (it directly sends push notifications from the app itself using a ready-made push notification provider (pushalert); I will implement the server-side features and my own notification server later).

**sort of, because my phone’s battery manager is aggressive asf. It kills any notification service when the device is locked. I've tried almost every solution from Don't Kill My App, but it’s still unreliable as hell. I might have to root my phone and make this a kernel function, or just switch phones all together.

I will post and update with server-side features soon.

Adios till then!

This got me thinking if there was a way to make a universal DB wrapper using LLMs, that can answer natural language queries on any database, given only the database connection string. I mean automatically parse and understand the schema and generate sql/no-sql commands for each query.

The basic principle is to first somehow give the language mode enough context about the schema of our database and then get it generate correct sql for a give natural language query.

I had already tried to do it with the earlier versions of gpt3.5 turbo but failed miserably. earlier versions of 3.5 turbo were instruct tuned versions of pruned gpt3 model. It kinda worked for extremely toy databases but quickly fell apart for any real use-case.

But right around june of this year, OpenAI introduced function calling. With it came a new instruct tuned model (gpt-4-0613) which adhered to the instructions better. This combined with 16k context length meant that it understood large database better due to larger context length and understood the natural language better to write sql queries on the data.

Now the remaining challenge was to make a sure that GPT always returned the correct results and not f*ck up the sql queries it generated. This task seems simple on surface but took a lot of experimentation to make it kinda viable.

To benchmark the viability I used my good old nalanda library database that I scraped during my junior year in college. It has about 10k rows of data, multiple tables and relationships and is about 300mb in size. I wrote about 20 queries of varying complexities and benchmarked run-time and accuracy of each query for every experiment(manually).

Here was the first viable experiment :

I extract out the relevant part of schema with examples :

db_params = {

}
import psycopg2

def fetch_database_info(db_params):
    result_str = ""

    connection = None
    try:
        # Establish a connection to the database
        connection = psycopg2.connect(**db_params)
        cursor = connection.cursor()
        
        # Fetch tables
        cursor.execute("""SELECT table_name FROM information_schema.tables WHERE table_schema = 'public';""")
        tables = cursor.fetchall()
        
        tables_str = "Tables:\n"
        for table in tables:
            table = table[0]
            tables_str += f"  - {table}\n"
            
            # Fetch columns for each table
            cursor.execute(f"""SELECT column_name, data_type FROM information_schema.columns WHERE table_name = '{table}';""")
            columns = cursor.fetchall()
            columns_str = "    Columns:\n"
            for column in columns:
                column_name, data_type = column
                columns_str += f"      - {column_name} ({data_type})\n"
                
            # Fetch sample data (optional)
            cursor.execute(f"SELECT * FROM {table} LIMIT 5;")
            sample_data = cursor.fetchall()
            sample_data_str = "    Sample Data:\n"
            for row in sample_data:
                row = [str(item)[:30] + ('...' if len(str(item)) > 30 else '') for item in row]
                sample_data_str += f"      - {row}\n"
            
            tables_str += columns_str + sample_data_str + "\n"
        
        result_str += tables_str
        
        # Fetch relationships (Foreign Key references)
        cursor.execute("""
        SELECT
            tc.table_name, kcu.column_name, 
            ccu.table_name AS foreign_table_name,
            ccu.column_name AS foreign_column_name 
        FROM 
            information_schema.table_constraints AS tc 
            JOIN information_schema.key_column_usage AS kcu
              ON tc.constraint_name = kcu.constraint_name
              AND tc.table_schema = kcu.table_schema
            JOIN information_schema.constraint_column_usage AS ccu
              ON ccu.constraint_name = tc.constraint_name
        WHERE tc.constraint_type = 'FOREIGN KEY';
        """)
        relationships = cursor.fetchall()
        relationships_str = "Relationships:\n"
        for rel in relationships:
            table, column, foreign_table, foreign_column = rel
            relationships_str += f"  - {table}.{column} -> {foreign_table}.{foreign_column}\n"
        
        result_str += relationships_str
        
    except Exception as e:
        return f"Error: {e}"
    finally:
        if connection:
            connection.close()
    
    return result_str

# Call the function
database_info_str = fetch_database_info(db_params)

print(database_info_str)

This gives me output similar to this :

Now to the gpt agent is quite simple :

Get natural language query from user > give GPT context about the db + the user query and generate sql from it > Validate and execute SQL > Give back the SQL error to GPT recursively till valid query is generated > Give output of that query + the input query to another agent to summarise > display the summary.

import subprocess
import psycopg2
import openai
import re
import schema

openai.api_key = ""

# Database connection parameters
db_params = {
}

def extract_sql(response):
    pattern = r'```sql\n(.*?)\n```'
    match = re.search(pattern, response, re.DOTALL)
    if match:
        return match.group(1).strip()
    else:
        return None

def get_valid_sql(messages, attempts=0):
    if attempts >= 5:
        return None, []
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        temperature=1,
        messages=messages
    )
    rawResponse = response.choices[0].message['content']
    sqlCommand = extract_sql(rawResponse)
    try:
        cur.execute(sqlCommand)
        rows = cur.fetchall()
        return rawResponse, sqlCommand, rows
    except Exception as e:
        conn.rollback()  # Rollback the current transaction
        messages.append({"role": "user", "content": "It gave me this error please correct it." + str(e)})
        return get_valid_sql(messages, attempts + 1)

def convert_to_natural_language(rows, original_query):
    messages = [
        {"role": "system", "content": "You are a helper that interprets database results into natural language. Convert the given tuples into a human-readable answer based on the user's input question."},
        {"role": "user", "content": f"Original Question: {original_query}"},
        {"role": "user", "content": f"Database Results: {rows}"}
    ]
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages
    )
    return "\n\n"+response.choices[0].message['content']

def fetch_db_schema():
    return schema.fetch_database_info(db_params)
    # """
    # Fetch the database schema: list of tables, columns, and their types.
    # """
    # try:
    #     cur.execute("""
    #         SELECT 
    #             table_name, 
    #             column_name, 
    #             data_type 
    #         FROM 
    #             information_schema.columns 
    #         WHERE 
    #             table_schema = 'public' 
    #         ORDER BY 
    #             table_name, 
    #             ordinal_position
    #     """)
    #     rows = cur.fetchall()
    #     schema_desc = ""
    #     current_table = ""
    #     for row in rows:
    #         table_name, column_name, data_type = row
    #         if current_table != table_name:
    #             schema_desc += f"\nTable: {table_name}\n"
    #             current_table = table_name
    #         schema_desc += f"  {column_name} ({data_type})\n"
    #     print(schema_desc)
    #     return schema_desc
    # except Exception as e:
    #     print("Error fetching schema:", e)
    #     return ""

def handle_user_query(messages):
    while True:  # Continue indefinitely until the user decides to exit
        query = input("\nWhats the query? (or type 'exit' to stop) ")
        if query.lower() == 'exit':
            break
        messages.append({"role": "user", "content": query})
        rawResponse, sqlCommand, rows = get_valid_sql(messages)
        if sqlCommand:
            print("\nExecuted SQL Command:", sqlCommand)
            messages.append({"role": "assistant", "content": f"SQL: {rawResponse}"})  # Add SQL to messages list
            # print(messages)
            natural_language_response = convert_to_natural_language(rows, query)
            print(natural_language_response)
        else:
            print("\nFailed to generate a valid SQL command after 5 attempts.")

# Connect to the database
conn = psycopg2.connect(**db_params)
cur = conn.cursor()

# Get database schema
schema = schema.fetch_database_info(db_params)

# Prepare initial messages for SQL generation
messages = [
    {"role": "system", "content": "you are a sql generator program. user inputs their db schema and they query in english. Your job is to think step by step and generate sql for that query to the best of your ability. Important !! : user input is case insensitive. wrap the sql part in '''sql ''' so that its easy for me to understand."},
    {"role": "user", "content": schema}
]

# Start handling user queries
handle_user_query(messages)

# Close the database connection
cur.close()
conn.close()

Extremely rudimentary but works on 16/20 of my test queries.

Next part is making it more robust for complex databases and making it modular to work on any given database string. I'll also share my benchmark and testing suite in the next part. I'll also be testing this(https://github.com/defog-ai/sqlcoder) OSS llm on my bench.

To be continued....

This is an active collection of resources I've found on understanding, training and fine-tuning, large language models.

I plan to update this often and it's much easier to do It in notion than on my blog so here's the notion link.

May your shoggoth be aligned with you 👾

So as for part 1 lets make a twitter scrape that scrapes all the handles of accounts a person follows. To help me make sense of this clusterfuck of an API I'm gonna use the divine powers of GPT4 .

Its honestly pretty surprising how good is gpt, at understanding the graph api twitter uses. It reduced the time to build this thing from like a weekend to like a couple of hours.Freakin magical.

This is the request twitter makes to make the initial fetch of the follower list :

Looking closely these are the variables encoded in the URL of the request :

{"userId":"183749519","count":20,"includePromotedContent":false}
{"blue_business_profile_image_shape_enabled":true,"responsive_web_graphql_exclude_directive_enabled":true,"verified_phone_label_enabled":true,"responsive_web_graphql_timeline_navigation_enabled":true,"responsive_web_graphql_skip_user_profile_image_extensions_enabled":false,"tweetypie_unmention_optimization_enabled":true,"vibe_api_enabled":true,"responsive_web_edit_tweet_api_enabled":true,"graphql_is_translatable_rweb_tweet_is_translatable_enabled":true,"view_counts_everywhere_api_enabled":true,"longform_notetweets_consumption_enabled":true,"tweet_awards_web_tipping_enabled":false,"freedom_of_speech_not_reach_fetch_enabled":false,"standardized_nudges_misinfo":true,"tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled":false,"interactive_text_enabled":true,"responsive_web_text_conversations_enabled":false,"longform_notetweets_rich_text_read_enabled":true,"responsive_web_enhance_cards_enabled":false}

In the second scroll of the list a variable called cursor gets added to the request which specified the sub-list of followers to fetch. This cursor is returned from the previous request as the timeline cursor.

{"userId":"183749519","count":20,"cursor":"1763689340909851626|1649001226504765398","includePromotedContent":false}

So to fetch all the list of followers of an account I just need to loop this request with different cursor till the beginning of cursor reaches 0. ie the request returns 0|xyz as the cursor value.

Now to build the parser I could study the whole response orrrrr I can just pass the resonse to GPT4 and ask it to build a parser for me 🤯.

The pagination code was Incorrect as i wasn't specific enough to what exact attribute to extract so I ask it to modify the code.

All of this worked the first try OFC. Now the only thing left was to loop this shit till the end of cursor. For this I gave it all the code it had previously written to modify and make the final script.

aaaand It worked. On the first attempt nonetheless lmao.

It was truly wild seeing it understand the api response so well.

I am planning to create an llm agent that automates all of this by just giving it a url of a webpage and a prompt about what things to scrape... but first lemme fix my twitter recommendations real quick.

If you, like me, haven't been following the state of the art in facial recognition, would't be aware that how GOOD these models have gotten over the years.

Experiments show that human beings have 97.53% accuracy on facial recognition tasks whereas some models like googles facenet have already reached and passed that accuracy level. (facet has 99.65% accuracy on Labeled Faces in the Wild  benchmark)

serengil has made an awesome wrapper for all the popular soda models like facenet, egg-face etc. called deepFace (not to be confused with facebook's deepface model).

First order of business is getting all the images from our database with all the enums.

We do so by using this simple command >

Next we'll need to convert these images to vector embeddings generated by the facenet mode. We'll use deepFace deepface.represent() method to do the same. It can take in base64 image as its input, which is perfect for our use case.

Then we package this data into a 1d vector and push into a representation list. faiss library we'll use in the future will required a 2d vector. So we'll use a simple for loop to get a vector of (8000,512) dims (facenet512 produces a array of 512 chunks).

Then we encode everything into a numpy array and save it on the disk. This way we won't have to generate the embeddings again when we deploy it to production later.

Next we pass this numpy array to the faiss library to index. Without faiss we would have to calculate the cosine distance between each vector when we search for a similarity match for a face. Faiss is pretty neat overall and allow us to run a nearest neighbour search that runs in millisecond even on gigantic multi-million vector datasets. check it out here.

To search for a match in an image we first convert the image to its vector representation then make a 3d np array and use index.search method to find the k most similar faces in the database.

Faiss return the index of the vector, we can pass this index to our representation vector to get the enum of the face.

And voila! we have made ourselves a super scalable surveillance system. Now to actually deploy this we'll first have to go over some trivial hurdles.

Here's the quick and dirty jupyter notebook showcasing what I described above.

In [ ]:

mName = "Facenet512"

In [ ]:

#Connect to DB

import psycopg2

try:
    conn = psycopg2.connect("dbname='nalanda' user='postgres' host='localhost' password='password'")
    print("connected")
except:
    print("I am unable to connect to the database")

In [ ]:

cur = conn.cursor()
        
cur.execute("SELECT enum,profile from raw limit 500")

rows = cur.fetchall()

len(rows)

In [ ]:

from deepface import DeepFace
import numpy as np

representation = []
failed_enums = []
for row in rows:
    try:
        vector = DeepFace.represent(row[1],model_name=mName)
    except:
        print("failed to process enum",row[0])
        failed_enums.append(row[0])
        continue
    pair = []
    pair.append(row[0])
    print(row[0])
    pair.append(vector)
    representation.append(pair)

embeddings = []
for i in range(0, len(representation)):
    embedding = representation[i][1]
    embeddings.append(embedding)

embeddings = np.array(embeddings, dtype='f')

np.save("embeddings",embeddings)

In [ ]:

import faiss                   # make faiss available
index = faiss.IndexFlatL2(embeddings.shape[1])   # build the index

In [ ]:

index.add(embeddings)
print(index.ntotal)

In [ ]:

testVector = DeepFace.represent(img_path = "test.jpg",model_name=mName)
target_representation = np.array(testVector, dtype='f')
target_representation = np.expand_dims(target_representation, axis=0)

In [ ]:

k = 20
distances, neighbors = index.search(target_representation, k)

representation[neighbors[0][0]][0]
# distances, neighbors

Connecting DB and Making it deployable to ☁️

In [ ]:

#Connect to DB

import psycopg2

try:
    conn = psycopg2.connect("dbname='nalanda' user='postgres' host='localhost' password='password'")
    print("connected")
except:
    print("I am unable to connect to the database")
cur = conn.cursor()

In [ ]:

import numpy as np
embeddings = np.load("all_embeddings.npy",allow_pickle=True)

In [ ]:

import faiss                   # make faiss available
index = faiss.IndexFlatL2(embeddings.shape[1])   # build the index

In [ ]:

index.add(embeddings)
print(index.ntotal)

In [ ]:

from deepface import DeepFace
mName = "Facenet512"
testVector = DeepFace.represent(img_path = "d.jpeg",model_name=mName)
target_representation = np.array(testVector, dtype='f')
target_representation = np.expand_dims(target_representation, axis=0)

In [ ]:

k = 5
distances, neighbors = index.search(target_representation, k)

ids = (int(neighbors[0][0]),int(neighbors[0][1]),int(neighbors[0][2]),int(neighbors[0][3]),int(neighbors[0][4]))

In [ ]:

cur.execute("select student.fname, student.lname from student join address on student.enum=address.enum join contact on student.enum=contact.enum join raw on student.enum=raw.enum join facenet on student.enum=facenet.enum where facenet.index=%s or facenet.index= %s or facenet.index= %s or facenet.index= %s or facenet.index= %s;",(ids))
cur.fetchall()

In [ ]:

cur.execute("select * from student join address on student.enum=address.enum join contact on student.enum=contact.enum join raw on student.enum=raw.enum join facenet on student.enum=facenet.enum where facenet.index=%s;",(int(neighbors[0][0]),))
cur.fetchall()

I created a new table with all the enums of the people whose images were available in the database with the index of those enums in the faiss database. Then it was just the game of converting input images to vectors and running a faiss search to get the nearest neighbor and find the enum of that index number in the DB and joinig it with the rest of tables to get the complete info.

Here's the modified notebook which does the same while making embeddings.

In [ ]:

mName = "Facenet512"

In [ ]:

#Connect to DB

import psycopg2

try:
    conn = psycopg2.connect("dbname='nalanda' user='postgres' host='localhost' password='password'")
    print("connected")
except:
    print("I am unable to connect to the database")

In [ ]:

cur = conn.cursor()
        
cur.execute("SELECT enum,profile from raw where LENGTH(profile) > 100 order by enum limit 10")

rows = cur.fetchall()

len(rows)

In [ ]:

from deepface import DeepFace
import numpy as np

index = 0 #representation length

representation = []
failed_enums = []
for row in rows:
    try:
        vector = DeepFace.represent(row[1],model_name=mName)
        cur.execute("INSERT INTO facenet (enum, index) VALUES (%s, %s)",(row[0],index))
        conn.commit()
    except:
        print("failed to process enum",row[0])
        failed_enums.append(row[0])
        continue
    pair = []
    pair.append(row[0])
    print(row[0])
    pair.append(vector)
    representation.append(pair)
    index+=1

embeddings = []
for i in range(0, len(representation)):
    embedding = representation[i][1]
    embeddings.append(embedding)

embeddings = np.array(embeddings, dtype='f')

np.save("embeddings",embeddings)

In [ ]:

import faiss                   # make faiss available
index = faiss.IndexFlatL2(embeddings.shape[1])   # build the index

In [ ]:

import numpy as np
embeddings = np.load("all_embeddings.npy",allow_pickle=True).shape

In [ ]:

index.add(embeddings)
print(index.ntotal)

In [ ]:

testVector = DeepFace.represent(img_path = "test.jpg",model_name=mName)
target_representation = np.array(testVector, dtype='f')
target_representation = np.expand_dims(target_representation, axis=0)

In [ ]:

k = 20
distances, neighbors = index.search(target_representation, k)

representation[neighbors[0][0]][0]
# distances, neighbors

In [ ]:

In part 2 I'll explain how I made a custom gradio frontend for it and pushed it to the cloud for ez inference.

Which lead me to creating this project.

> Getting the endpoints from truecaller web-app

After logging into the web-app the tc server generates a session token which is used to AUTH subsequent requests.

The problem I ran into was that truecaller only allows the lookup of 10 numbers/day on its website. For anything more than this you have to use their mobile app. So onto the next step..... breaking their andorid app.

> Reverse engineering the android app

We can use tools like burpsuite proxy to intercept normal requests on any device (android, iOS etc.) by just installing a custom ca certificate onto the device to trust and routing all the traffic through the proxy.

Here's how to do it with burp.

But modern apps use something called SSl-pinning. This allows them to use their own ssl certificate store instead of trusting the device's own certificates.

Detailed writeup here.

So if we want to intercept network requests of an SSL pinned app we must first find a way to disable it or add our own certificate to the app's trusted cert store. This involves roughly these steps ->

• Decompile the app using apk-tool to generate smali code for the app.

• Remove all the certificate verification checks from the code (actually hard depending on implementation)

• Recompile the app and sign it.

Luckily shroudedcode has made an awesome cli app to do just that.

This method may or may not work depending on the app's version as its implementation of ssl-pinning may differ from what apk-mitm aims to patch. I reccomend trying out any older version of the app if the tool fails on the newer one.

Another frequently used technique used by app-sec professionals is frida. Which is described best as greasy monkey for andorid apps. If you don't have a rooted device for frida runtime. You can use a sub-framework called objection to embed frida runtime in the app itself.

Last technique to disable cert-pinning, which is also my favorite one so far is using good old xposed framework with justtrustme module. This disables ssl-pinning across the board for the whole device.

Setting up environment ->

I went with bluestacks instead of avdmanager because it didn't want the bloat(android studio) that comes with it.

After installing bluestacks, I recommend this tool to root the vm and install xposed framework on it with minimal hassle.

After doing that, use oxyproxy to proxy to our burpsuite machine. and install the burpsuite ca certificate using this app. After doing all of that just install the truecaller app on your vm and you are ready for some interception.

I used a fake mobile number from some sketch sms-recieve site to register for the app.

Also, If for some reason the verification fails and you are blocked for x amount of hours before retrying, just change the android-id and imei fom the bluestacks-tweaker's device section.

This is the request that the app sends to log in a user ->

Once successful it returns a session token which is valid till you logout of the app.

Now when you try to search for a number it generates this request ->

This returns a json response with plethora of information. Everything form public email addresses to profile pics to social media links.

Mission successful ! now all we gotta do is replicate this request and give it a better front end.

Making a telegram bot out of it ->

Wrapping all of this up I replicated this request in go and used this library to make a telegram bot out of it.

Here's the github gist for the same. Just replace the session token and bot auth token with your session token and bot token and you are good to go.

Some notes ->

• Officially tc limits the free accounts to 100 lookups a month, but I was able to do much more than that. IDK how...

• I am just marshalling the json responses into a string, and pretty-printing to the telegram chat {too lazy to make gigaton of structs and massaging the json o/p)

Having watched David Fincher's fine fictional work. I put on  Atticus Ross's and Trent Reznor's music on loop and wrote a facemash clone in a plan to get rusticated from my college.

forking off main database ->

first off I forked out my postgres database using the following command in order to avaoid performing triple joins every time a picture gets rated.->

and added some more columns to it ->

the algo ->

Giving in to following fincher's work I went with elo rating system instead of using better options.

Here's how this algorithm calculates the probability of a player winning the match-up ->

Here's the formula for final rating of the player which won ->

Here's my golang implementation ->

i am using this function to test the various values of k,d and s to see, after how many number of losses the score of a player went negative.Then adjusted the base rating accordingly to a reasonable 200-ish match-ups till negative.

making the website :

onto the worst part. making the front end. Opting the laziest way out, I used the example code from material bootstrap library, modified it and used go's inbuilt templating engine to make the rankings and profile page.

wrangling database :

using the following sql query I fetched two random images with their enums from the DB to hydrate the page every-time it was refreshed.

and this query to get the top 15 for the rankings page ->

after each vote the /vote endpoint is called with winner and looser enums which updates the database with the new ranking of both winners and losers according to the elo algorithm. I'll need to add rate limiting to prevent abuse if I ever decide to release this app to a wider audience (which is never gonna happen).

Finished result ->

So I decided to to just that. Scraping data of current (and past) students studying in the college. All in good faith (obviously).

Here was the plan ->

generating enums :

I don't know what the complete roll numbers of all the students are. But I do know what first 4 digits represent.

If I could brute-force the rest of the 5 digits without breaking or alerting the library servers I could hypothetically scan for the successful hits and scrape the data.

So I wrote this program that takes the year of students as input and generates all the permutations of the roll numbers. Then stores it in a Postgres table with the following fields.

We'll use this table as a reference of the successful hits and the the enums not yet touched.

reverse engineering the login flow :

Next logical step is figuring out the api endpoints and see how server authenticates the requests one logged in.

In this case one you submit the used id and password, it called a perl script which sends back session token as cookie header if you were authenticated successfully.

The gotcha in this login flow is that the server still sends a 200-Ok request with with a session cookie even if you weren't logged successfully.

So to figure out if the uid-password combo was correct, I had to see what was inside the request, parse it and see if I was logged in successfully.

This snippet of code does just that. It scans the 45 character of the response data and checks of it a particular character. If yes, then classifies this as a successful hit and stores the session token.

sending 500k request's to the server concurrently without raising any red flags :

the problem with using brute-force to find out successful enum hits is that every request take at least 1.4 seconds to complete (even on local network) and 2+ seconds if it is a successful hit. This is server side latency and there;s nothing I can do about it.

I however can uses go's robust inbuilt concurrency model to send a lot of requests concurrently . This presents the risk of dosing the library's server and getting my IP blocked.

The solution of all of this is using a concurrency pattern called worker pools. Worker pools uses go's inbuild channels to build a jobs queue from which my (limited) pool of goroutines can concurrently fetch jobs to do.

This video explains all of this and more pretty lucidly.

This snippet of code makes two separate channels, creates n number of go routines and then fills the jobs channels with data it fetched from the permutation's db. At last it waits for the workers to finish their jobs by listening to the results channels.

parsing all the data from the profiles page :

each student has a my profile page in which they can change their personal details all of this comes pre-hydrated as an HTML page and NOT as a json response which means I had to write my own parser.

Go's inbuilt html parsing engine is way too complicated for my wrap my head around in the time-frame of this project. So I went with an awesome library called goquery which implements jquery in native go and is used by other major scraping libs like colly.

The parser I wrote is way too ugly and and duck-taped together to be discussed in this post but ye can have a look at it here.

handling images :

to scrape images I wrote a separate module which requested the png (<60kb) form the server, converted it into base64 url string compatible with chrome and chucked it into my database.

Other notes :

for my database I went with postgres as its awesome and I had a some experience writing highly concurrent go apps with it. It does not disappoint or fuck-up ever unlike its nosql counterparts.

I was able to run ~50 gophers concurrently without experiencing some sort of throttling.

Modularising the apps smary can save you and others tonns of headches and makes the overall experice of working on an project more fun.

I still don't how how to write good tests form my code.

final result :

this was the final size of the db after hitting the library server 500k times scraping data of more than than 8000 students.

This project's source is open on my github.

• Authentication and authorisation using jwt tokens.

• Customizable sessions where I can set a particular time for jwt token expiry (one of my labs had restrictions on opening incognito tabs so this removes the need to open incognito again)

• CRD -> I purposefully removed update feature for better consistency when sharing (I'll get to this point later).

• CORS support and CSRF protections.

• micro (<15mb) file storage solution.

I decided to go with cloudflare workers as I've used their other services before and I really like the company. Also I wanted to see what can workers KV actually so I've tried incorporating all of its features into it.

Building a cf project is pretty simple, you just need to install wrangler cli and login into your cf account. The cli will guide you through the rest of the process.

It will generate a project structure kind of like this ->

The main worker script will reside in the index.js file and the worker configurations will reside in the wrangler.toml file.

My tooling file looks like this ->

Here name is the name of project and type is webpack. cf supports(officially) building workers in rust(compiled using wasm) , pure javascript or using a bundler like webpack. Do note not all node compatible package are cross compatible with cloudflare, especially the once using node's standard library. Because unlike other server-less solutions workers run on bare isolated v8 instances instead of spinning up a node container every time a request comes through. This effectively eliminates the cold start problem I've faced with gcp .

kv namepaces variable hold our namespaces ids and their aliases along with the preview id's reuired when running the workers dev server.

The [vars] are the global environment variables which get stored in the cloudflare's secret store. I have my main passkey for the pastebin and my jwt signature stored here. It recommended to use a thrid paarty secrets manager if you are deploying it in production tho.

You can install the npm packages using the standard npm commands and then use wrangler build command to build the project.

wrangler dev -> spins up a test server for locally testing the workers

wrangler publish -> builds and publishes the workers on the cf web.

So lets look at our index.js file . Interestingly most of it (minus cors) was written in just under 4 hours and its barely 150 LOC for the WHOLE api !

import { Router } from 'itty-router'
const jwt = require('@tsndr/cloudflare-worker-jwt')

const router = Router()

//Csrf Headers -> change before prod
const myHeaders = new Headers();
myHeaders.set("Access-Control-Allow-Origin", '*');
myHeaders.set("Access-Control-Allow-Headers", '*');
myHeaders.set("Access-Control-Allow-Methods", "GET,HEAD,POST,OPTIONS");
myHeaders.set("Access-Control-Max-Age", "86400");
// myHeaders.set("Access-Control-Allow-Credentials", "true")


//Root Page
router.get("/", async () => {
  myHeaders.set("content-type", "text/html")
  return new Response(html, { status: 200, headers: myHeaders })
})

//Cookie Parser
function getToken(request) {
  var list = {};
  const cookieHeader = request.headers.get("Cookie")
  if (!cookieHeader) return "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c" //bandaid plz fix before prod
  cookieHeader.split(';')
    .map(v => v.split('='))
    .reduce((acc, v) => {
      acc[decodeURIComponent(v[0].trim())] = decodeURIComponent(v[1].trim());
      list = acc
    }, {});
  const token = list["passkey"]
  return token
}

//Pass all the csrf preflight -> chage before prod
router.options("*", async () => {
  return new Response("OK", { status: 200, headers: myHeaders })
})

//Jwt generator
router.get("/authgen", async request => {
  const pass = request.headers.get("passkey")
  const exp = request.headers.get("expires")
  if (pass == passkey) {
    const token = await jwt.sign({
      name: 'doom slayer',
      email: 'okdoomer@lol.com',
      exp: Math.floor(Date.now() / 1000) + (parseFloat(exp) * (60 * 60)) // Expires: Now + Xh
    }, signature)
    myHeaders.set('Set-Cookie', `passkey=${token}; Path=/ ;Max-Age=${(parseFloat(exp) * (60 * 60))}; Secure; HttpOnly;`)//add  SameSite=None to cookie props for cors "stuff"
    return new Response(token, { status: 200, headers: myHeaders })
  }
  else {
    return new Response("401, wrong passkey", { status: 401, headers: myHeaders })
  }
})

//view a single post
router.get("/view", async request => {
  const token = getToken(request)
  const isValid = await jwt.verify(token, signature)
  if (isValid == true) {
    const pid = request.headers.get("pid")
    const paste = await pastebin.get(pid);
    return new Response(paste, { status: 200, headers: myHeaders })
  }
  else if (valid == false) {
    return new Response("401, wrong passkey", { status: 401, headers: myHeaders })
  }
})


//list all posts
router.get("/list", async request => {
  const token = getToken(request)
  const isValid = await jwt.verify(token, signature)
  if (isValid == true) {
    const value = await pastebin.list()
    const json = JSON.stringify(value)
    myHeaders.set("content-type", "application/json;charset=UTF-8")
    return new Response(json, { status: 200, headers: myHeaders })
  }
  else {
    return new Response("401, token expired or invalid", { status: 401, headers: myHeaders })
  }
})

//view a shared post
router.get("/share/:id", async ({ params }) => {
  let input = params.id
  const paste = await publicbin.get(input);
  return new Response(paste, { status: 200, headers: myHeaders })
})

//create a new post
router.post("/create", async request => {
  const token = getToken(request)
  const isValid = await jwt.verify(token, signature)
  const body = await request.formData()
  const ct = body.get('content')
  const title_ = body.get('title')
  let flag = body.get('public')
  if (flag == null) {
    flag = "0"
  }
  const ts = Date.now()
  if (isValid == true && ct != null) {
    await pastebin.put(ts, ct, { metadata: { title: title_, timeStamp: Date.now(), isPublic: flag } })
    if (flag == "1") {
      await publicbin.put(ts, ct, { metadata: { title: title_ } })
    }
    return new Response(Date.now(), { status: 200, headers: myHeaders })
  }
  else {
    return new Response("401, wrong passkey or invalid request idk", { status: 401, headers: myHeaders })
  }
})

//Delete a post
router.get("/delete", async request => {
  const token = getToken(request)
  const isValid = await jwt.verify(token, signature)
  const pid = request.headers.get("pid")
  let isPublic = request.headers.get("isPublic")
  if (isPublic == null) {
    isPublic = "0";
  }
  if (isValid == true && pid != null) {
    if (isPublic == "1") {
      await publicbin.delete(pid)
    }
    await pastebin.delete(pid)
    return new Response("done", { status: 200, headers: myHeaders })
  }
  else {
    return new Response("401, wrong passkey or invalid request idk", { status: 401, headers: myHeaders })
  }
})

//404
router.all("*", () => new Response("404, not found!", { status: 404, headers: myHeaders }))

//Connect to router
addEventListener('fetch', (e) => {
  e.respondWith(router.handle(e.request))
})


addEventListener("fetch", event => {
  event.respondWith(handleRequest(event.request))
})

The code is kind of crude with some band-aids here and there. You should probably configure cors and csrf stuff as your liking. I've given a rough template on how it should happen.

I've used two namespace for kv store. One of all the private+public stuff and one for exclusively public stuff, which helps me separating the public facing - no auth side of my api more efficiently.

I've used the itty-router library for basic routing and cloudflare-worker-jwt library for generating and validating jwt tokens . both of these are specifically written for cf workers.

I also wrote a super sketchy frontend using materialise which took me more then 6 hours to finish and reminded me why I only do backend these days.

The frontend is served through the root route "/" using the worker itself thus making my cors pains a little bit better (and believe me cors is a yuuge PITA esp if you're working on compliance).

Now storing files is kinda tricky. I thought about adding dropbox/gdrve api to the worker but building such system without using o-auth is an uphill battle. I've used my previous pastebin at some extremely sketchy locations/cyber-cafes so I need a robust way of storing and accessing files quickly. Also my upload requirements are pretty basic (its mostly code I wrote someplace else, pdfs or word files).

I landed up on this solution -> encode the files in base64 url string -> chuck it into database -> use browser's built in decode function to decode and download the file -> (optional: for larger files, shard the url in chunks and store it in different kv's with its access table written in keys metadata making a adhoc linked list)

workers kv supports the value size of upto 25MB.. keeping in mind the 33% penalty for base64 encoding, it gives me 16.75MB of file storage per record which is plenty for my current use case. I may add shardng in a future update.

ggs till then

ps: here's the demo

First thing that I wanted to do was replace python with some compiled language. Python is a great language for lightening fast prototyping but its interpreted nature makes it terrible for high-performance applications. This bundled with the difficulty to package/ship it and make it cross compatible - is a a huge pain in the culo. Believe me I've tried everything from freezing to virtual environments to py2exe. I just could not get the packaging right.

In the end I had two options on my plate ..... GoLang and C++. I had more experience building stuff in C++ than go, so I knew how much pain in the ass it would to write somewhat decent c++ code. So decided to give golang a try and honestly.... my dev journey with it was better than I expected. I feel like the go has best of both worlds. Flexibility, prototype(ability?) , and readability of python and performance and bundled/compiled nature of c++.

Second I wanted to reduce the latency. Like REALLY eliminate it. Because I have a project in mind to use this for custom bindings in doom eternal and if you lag even for a 100ms on you UN run its probably run over for you. I tried using http requests - which were slow as a turtle with 50-60ms average response times. Tried using MQTT(the protocol) - which was supposed to be fast but still hung around at 20-25ms ping (I have no clue why. Maybe somewhere I fucked it up somewhere in my implementation of it.... not sure.... maybe I'll retry using MQTT in the future). In the end I decided to give pain old TCP sockets a try and here were the results -> average latency over 10000 requests -> 5-7ms average response time. :D

Compare this to the flagship wireless HID technology in the market - Logitech lightspeed wireless - coming in at 1ms.... My iteration was not too shabby. Its good enough for my TAG1 UN run.

Third thing was battery life...... Which I still haven't figured out yet. ESP consumes around 100mA during normal mode. When couple with a single AAA battery with a boost converter gives about ~8 Hours of use time. (Compare this to the 250 hours I get on my g305). WIFI(esp) consumes wayyyyyy too much power for a portable battery powered device. Which is the reason I guess why Wi-Fi HID devices haven't been popular. Stay tuned for part 3 when/if I figure it out to make this thing last longer either by using a different SOC or using sleep and interrupts on esp.

Here's the source....

I am planning to create a go HID library to support precise thumbstick/mouse inputs for CAD applications. Now that I can achieve sub 10ms this shouldn't be impractical. So stay tuned for that. GGs till then.

Anyways.... I recently (last night) built an OSINT tool to save instagram stories to dropbox anonymously. Here's how it works ->

• Use headless chrome to visit this page -> 'https://www.anonigviewer.com/profile.php?u='+userName . Why use a third party website ? Because I am lazy asf and was trying to speedrun this project. moving on....

• Find x-paths of all the latest stories and extract the media uris using almighty regex.

• Download the media using requests whilst ignoring all the ssl warnings. (don't do this in prod)

• Maintain a local database of the stories you've downloaded with timestamps and stuff.

• Upload all of the media you ripped to dropbox using their awesome api (seriously, they have the best api-docs I've read in a while).

Here's the code ->

from selenium import webdriver
import time
import re
from selenium.webdriver.common.by import By
from selenium import webdriver
import time
import re
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import requests
import dropbox
from tinydb import TinyDB, Query

# Declare yo user
user = 'doom'
storeFolder = "sauce/"

driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://www.anonigviewer.com/profile.php?u='+user)

# Upload to dropbox
uriList = []

def dropUpload(uris):
	dropbox_access_token= "" #Yo acess token goes here
	client = dropbox.Dropbox(dropbox_access_token)
	print("DataStore Connected")
	for uri in uris:
		to_path= "/"+uri
		from_path=uri
		client.files_upload(open(from_path, "rb").read(), to_path)
		print("Upload Komplete")


# Poor man's nosql store omegalul

def isPresent(url):
	db = TinyDB('db.json')
	p = Query()
	present = db.search(p.link==url)
	if(len(present)==0):
		db.insert({'link': url, 'timestamp': time.time()})
		return False
	else:
		return True

# Coutesy Of StackOverFlow lol

def download_file(url,type):
	if(isPresent(url)==False):
		timestr = time.strftime("%Y%m%d-%H%M%S")
		local_filename = storeFolder+timestr+type
		r = requests.get(url,verify=False, stream=True)
		with open(local_filename, 'wb') as f:
			for chunk in r.iter_content(chunk_size=1024): 
				if chunk:
					f.write(chunk)
		uriList.append(local_filename)
		return local_filename

# Ugly code don't look

wait = WebDriverWait(driver, 50)	# 50 Sec of summer
sauceCheck = wait.until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[4]/div[3]/div/div[1]/div[1]/div/div')))
sauces = driver.find_elements_by_class_name('mb-4')
print(len(sauces))
i=1
preSauces = []
for sauce in sauces:
	preSauces.append(driver.find_element_by_xpath('/html/body/div[4]/div[3]/div/div[1]/div['+str(i)+']/div/div').get_attribute("onclick"))
	i=i+1

driver.close()

# Video or an Image ?

for preSauce in preSauces:
	if preSauce.find("GraphVideo")!=-1:
		type_ = ".mp4"
	elif preSauce.find("GraphImage")!=-1:
		type_ = ".jpg"
	link = re.findall(r'(https?://\S+)', preSauce)[0].split('"')[0]
	download_file(link,type_)

dropUpload(uriList)

Couple of things to take care of ->

Dropbox requires to start directory names with "/" . Make sure to you give appropriate write permissions in the app console.

Now just package this script -> add it to a cron-job on a remote server and bam. Speedrun insta stalker any% complete xD.

Btw I am finishing up working on a really cool low-level driver written in go. I'll post something about it soon. GGs till then.

edit1: Here's some modified code if you want to run it on an ubuntu server :

from selenium import webdriver
import os
import time
import re
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import requests
import dropbox
from tinydb import TinyDB, Query
from selenium.webdriver.chrome.options import Options

# Declare yo user
user = 'public user goes here'
storeFolder = "sauce/"

#Chrome Config
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox") # linux only
chrome_options.add_argument("--headless")


driver = webdriver.Chrome(options=chrome_options)
driver.get('https://www.anonigviewer.com/profile.php?u='+user)

# Upload to dropbox
uriList = []

def dropUpload(uris):
        dropbox_access_token= "access token goes here"
        client = dropbox.Dropbox(dropbox_access_token)
        print("DataStore Connected")
        for uri in uris:
                to_path= "/"+uri
                from_path=uri
                client.files_upload(open(from_path, "rb").read(), to_path)
                print("Upload Komplete")


# Poor man's nosql store omegalul

def isPresent(url):
        db = TinyDB('db.json')
        p = Query()
        present = db.search(p.link==url)
        if(len(present)==0):
                db.insert({'link': url, 'timestamp': time.time()})
                return False
        else:
                return True

# Coutesy Of StackOverFlow lol

def download_file(url,type):
        if(isPresent(url)==False):
                timestr = time.strftime("%Y%m%d-%H%M%S")
                local_filename = storeFolder+timestr+type
                # NOTE the stream=True parameter
                r = requests.get(url,verify=False, stream=True)
                with open(local_filename, 'wb') as f:
                        for chunk in r.iter_content(chunk_size=1024):
                                if chunk: # filter out keep-alive new chunks
                                        f.write(chunk)
                uriList.append(local_filename)
                return local_filename

# Ugly code don't look

wait = WebDriverWait(driver, 50)        # 50 Sec of summer
try:
        sauceCheck = wait.until(EC.visibility_of_element_located((By.XPATH, '/html/body/div[4]/section[3]/div[1]/div/div[1]/div[1]/div/div'))) #/html/body/div[4]/section[3]/div[1]/div/div[1]/div[1]/div/div
except:
        driver.close()
        os.system("sudo killall chromedriver")
        exit()
sauces = driver.find_elements_by_class_name('mb-4')
print(len(sauces))
i=1
preSauces = []
for sauce in sauces:
        preSauces.append(driver.find_element_by_xpath('/html/body/div[4]/section[3]/div[1]/div/div[1]/div['+str(i)+']/div/div').get_attribute("onclick"))
        i=i+1

driver.close()

for preSauce in preSauces:
        if preSauce.find("GraphVideo")!=-1:
                type_ = ".mp4"
        elif preSauce.find("GraphImage")!=-1:
                type_ = ".jpg"
        link = re.findall(r'(https?://\S+)', preSauce)[0].split('"')[0]
        download_file(link,type_)

dropUpload(uriList)

It stores the text in a public database, fully encrypted and decrypts it on the client side using credential stored on client's browser.

Here's some crucial code samples and explanations -

var CryptoJS = require("crypto-js");
import Cookies from 'js-cookie'
var fb = require('./store.js');

if (Cookies.get('key_')) {
    var unlocked = true;
    var key_ = Cookies.get('key_');
} else if (!Cookies.get('key_')) {
    var unlocked = false;
    var key_ = null;
    alert("¯\_(ツ)_/¯")
}
var inputField_;
export default {
    name: 'App',
    beforeCreate() {
        fb.db.collection("bin").doc("paste")
            .onSnapshot(function (doc) {
                inputField_ = decrypt(doc.data().field);
                this.inputField = decrypt(doc.data().field);
            });
    },
    updated() {
        inputField_ = this.inputField
    },
    data() {
        return {
            unlocked_: unlocked,
            inputField: inputField_,
            password: null,
        }
    },
    methods: {
        unlock: function (event) {
            if (checkKey(this.password)) {
                Cookies.set('key_', this.password, {
                    expires: 365
                })
                if (this.password) {
                    this.unlocked_ = true;
                    this.decrypted = true;
                }
            }
        },
        save: function (event) {
            // Add a new document in collection "cities"
            fb.db.collection("bin").doc("paste").set({
                    field: encrypt(this.inputField)
                })
                .then(function () {
                    console.log("Document successfully written!");
                    window.navigator.vibrate(200);
                })
                .catch(function (error) {
                    console.error("Error writing document: ", error);
                });
        },
        update: function (event) {
            this.inputField = inputField_
        }
    },
    computed: {
        shit: function () {
            alert(inputField_)
            return inputField_;
        }
    }
}

function decrypt(data_) {
    if (key_) {
        try {
            var decrypted = CryptoJS.AES.decrypt(data_, key_);
            return decrypted.toString(CryptoJS.enc.Utf8);
        } catch (e) {
            return data_
            console.log(e)
        }
    } else {
        return data_
        console.log(e)
    }
}

function encrypt(data_) {
    if (key_) {
        var encrypted = CryptoJS.AES.encrypt(data_, key_);
        return encrypted.toString();
    } else {
        return data_
        console.log(e)
    }
}

function checkKey(input_) {
    var decrypted = CryptoJS.AES.decrypt('U2FsdGVkX19CZxyjXIRZeb3N+ffdvboRFIyQELM1ab0=', input_);
    if (decrypted.toString(CryptoJS.enc.Utf8) == "aezakmi") {
        return true;
    }
}

• The app uses vue.js so this is the script part of the app.vue file.

• It uses a javascript library - cryptojs to encrypt and decrypt the text on the browser side .

• First it checks for the decryption key in the browser cookies and throws out the error for the uses to enter the key. The key is stored in the browser for 1 year.

• decrypt() function decrypts the aes256 strings obtained from public database.

• encrypt() function encrypts the text to aes256 string using the key from cookies.

• the checkKey() function is used to check if the entered key is correct or not by verifying if the entered key decrypts a predetermined text.

• They whole thing uses firebase for a database WITHOUT any public rules.

• As the data is encrypted to the highest(actually 2nd highest) standard before storing , any 3rd party won't be able to see the data (unless they are able to steal your browser cookies).

This is the final result ! >>

Here's the complete app.vue file for reference. New suggestions are welcome ... Enjoy!

Here's some key point I liked from the article -

• Every Western institution was unprepared for the coronavirus pandemic, despite many prior warnings.

•  And that is a failure of action, and specifically our widespread inability to *build*.

• A government that collects money from all its citizens and businesses each year has never built a system to distribute money to us when it’s needed most.

• You don’t just see this smug complacency, this satisfaction with the status quo and the unwillingness to build, in the pandemic, or in healthcare generally. You see it throughout Western life, and specifically throughout American life.

• Why not educate every 18 year old? Isn’t that the most important thing we can possibly do? 

• We know how to build highly automated factories. We know the enormous number of higher paying jobs we would create to design and build and operate those factories.

• Is the problem capitalism? I’m with Nicholas Stern when he says that capitalism is how we take care of people we don’t know — all of these fields are highly lucrative already and should be prime stomping grounds for capitalist investment, good both for the investor and the customers who are served.

• The problem is inertia. We need to want these things more than we want to prevent these things.

• The right must fight hard against crony capitalism, regulatory capture, ossified oligopolies, risk-inducing offshoring, and investor-friendly buybacks in lieu of customer-friendly (and, over a longer period of time, even more investor-friendly) innovation

• Milton Friedman once said the great public sector mistake is to judge policies and programs by their intentions rather than their results. Instead of taking that as an insult, take it as a challenge — build new things and show the results!

• Why can’t 100,000 or 1 million students a year attend Harvard? Why shouldn’t regulators and taxpayers demand that Harvard build? Solve the climate crisis by building.

• We need to demand more of our political leaders, of our CEOs, our entrepreneurs, our investors. 

• We’re all necessary, and we can all contribute, to building.

• Every step of the way, to everyone around us, we should be asking the question, what are you building?

• If the work you’re doing isn’t either leading to something being built or taking care of people directly, we’ve failed you, and we need to get you into a position, an occupation, a career where you can contribute to building. 

• There are always outstanding people in even the most broken systems — we need to get all the talent we can on the biggest problems we have, and on building the answers to those problems.

The last paragraph was the most moving one -

Our nation and our civilization were built on production, on building. Our forefathers and foremothers built roads and trains, farms and factories, then the computer, the microchip, the smartphone, and uncounted thousands of other things that we now take for granted, that are all around us, that define our lives and provide for our well-being. There is only one way to honor their legacy and to create the future we want for our own children and grandchildren, and that’s to build.

Marc Andreessen

Now I wanted to control my systems over the wifi. SSH or telnet would't suffice as I NEEDED mouse input. (Basically wanted to make a wifi enabled scroll wheel). This problem gave birth to this project.

Basic idea -

• Set up a local server to get requests from the local network.

• Use esp to send get request to the server. Request may include specific data (like mouse coordinates or keystrokes) or just Boolean data to tell system to perform a specific function.

• Every time a server gets a request, perform the specific function by using appropriate system level commands to send hid events to system.

• Reduce the input lag as much as possible by maintaining a constant connection between server and the esp.

1. Setting UP the server :

To server the requests I decided to use FLASK - A lightweight-micro web framework written in python. I selected it for two reasons - 1) I have used it before(duh!) , 2) Its laughably simple to get up and running , 3) Its based on python so I can use plethora of libraries to interface with the I/O.

Here's the basic setup ->

import flask

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/', methods=['GET'])
def home():
    return "<h1>SUP!<h1>"

app.run(host = '0.0.0.0')

Only important thing to note here is that we are running it at 0.0.0.0 instead of localhost to make it visible to the whole network network.

2. Generating hard-coded HID events :

To send keystrokes or mouse input we are going to use pyautogui . It an amazing cross platform library which hides all the complexity in interacting with different de(s) for different os(es). To quote from its official github page :

On Windows, PyAutoGUI accesses the Windows API (also called the WinAPI or win32 API) through the built-in ctypes module. 
On macOS, PyAutoGUI uses the rubicon-objc module to access the Cocoa API.
On Linux, PyAutoGUI uses the Xlib module to access the X11 or X Window System.

Install it on your system using :

pip install pyautogui
#for linux only
sudo apt-get install python3-xlib

Try it by running the following on your system :

import flask
import pyautogui 

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/up', methods=['GET'])
def home():
    pyautogui.scroll(-20) 
    return "<h1>Scrolling Up<h1>"

app.run(host = '0.0.0.0')

After running this script(in su mode!) go to your pc's local ip address on your default port (in my case it 192.168.1.78:8888) and see if reloading the page sends scroll down command to the system.

If it works good !. If it doesn't go here to troubleshoot.

3. Sending get requests using esp :

To do this first you have to connect your esp module to your local wireless network, detect hardware input and send send the get request to the server's address.

Here's the code :

#include <ESP8266WiFi.h>

#include <ESP8266HTTPClient.h>

const char * ssid = "ssid";
const char * password = "pass";

void setup() {

  Serial.begin(115200);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {

    delay(1000);
    Serial.print("Connecting..");
  }

}

void loop() {

  if (WiFi.status() == WL_CONNECTED) { //check connection

    HTTPClient http;
    if (digitalRead(BUTTON) == LOW) {
      http.begin("DESTINATION"); //request destination
      int httpCode = http.GET();
      if (httpCode > 0) {
        String payload = http.getString();
        Serial.println(payload); //response
      }
      http.end();
    }
  }
}

Replace "ssid" , "password" with your wifi details. Replace "DESTINATION" with the address you entered in your browser in the last step. Replace (BUTTON) with the pin number your connected your trigger to.

4. Sending KeyStrokes/Dynamic data :

To receive dynamic in flask change -

@app.route('/', methods=['GET'])

to -

@app.route('/<stroke>', methods=['GET'])

To send keystrokes use :

pyautogui.typewrite(stroke, interval=0)

Refer to this document to get into details.

Here's the final code :

import flask
import pyautogui 

app = flask.Flask(__name__)
app.config["DEBUG"] = True

@app.route('/<stroke>', methods=['GET'])
def home():
    pyautogui.typewrite(stroke, interval=0)
    return "<h1>done<h1>"

app.run(host = '0.0.0.0')

Now your can send keystokes to your system by editing the url after you ip (eg: http://192.168.1.78:8888/hello)

This is it for part one. In part two i will be assembling this shit into a neatly structured library with all the hid options from joysticks to scroll wheels, and game controllers. I will also try to reduce the lag/latency between hardware button presses and actual input by using something better to send requests .

So as a trial I decided to make financial bots for telegram using python and bs4.

bot_1 : StonkBoi

This bot scrapes the live stock data of a company from yahoo finance and reports back on telegram. It uses bs4 and requests to search and scrape the data and telegram python library to get the commands and send the info.

Bot is divided into two files the first file dataFetch.py has all the scraping goodness organised into different functions which are called in the tradeBot.py file.

Here's some sample code for the dataFetch.py :

def lastTradedPrice(ticker):
    page = requests.get("https://in.finance.yahoo.com/quote/"+ticker)
    soup = BeautifulSoup(page.content, 'html.parser')
    val = soup.findAll("span", {"class": "Trsdu(0.3s) Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(b)"})
    if len(val) >=1 :
        return val[0].get_text()
    else:
        return "Invalid Request"

Here function lastTradedPrice request the whole yfinance page for the given ticker and then beautiful soup fetches the price element using its findAll function to filter for specific class( in this case : Trsdu(0.3s) Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(b)). The function returns "Invalid request" it it detects the value is null . We cannot use the response code of the requests function as the page still returns the not found page and send the 200 status code.

Here's the complete source code for the bot.

Here is the list of commands :

Enter '$<ticker> x' where x can be 
sum => summary 
ltp => Last Traded Price 
mktcap => Market Cap 
chg => Day's price change 
prevclose => Previous Close Price 
openprice => Opening Price 
dayrange => Day's price range 
52range => 52 Week Range
vol => volume
pe => P/E Ratio 
div => Dividend Yields 
 Example => $SPY sum 
 Example-2 => $HDFCBANK.NS ltp

bot_2 : MarginsBoi

This bot fetches the latest margins info provided by my stonk broker - Zerodha. It can find latest margins on commodities, equities, equity futures and currency futures.

The scraping part of the bot is also accessible through command line/terminal.

Here's some sample code of the scraping part :

def equityFutures(ticker):
    j =0
    page = requests.get("https://zerodha.com/margin-calculator/Futures/")
    soup = BeautifulSoup(page.content, 'html.parser')
    table = soup.find("table", {"class": "data futures"})
    table_rows = table.find_all('tr')
    head = 'Contract\tExpiry\tLot Size\tPrice\tNRML Margin\tNRML Margin Rate \tMWPS\n\n'
    data = ''
    for tr in table_rows:
        if j>=1:
            td = tr.find_all('td')
            row = [i.text for i in td]
            completeRow = ''
            if ticker.lower() == row[1].lower().replace('\n', '').replace('\t', ''):
                for index, cell in enumerate(row, start=0): 
                    if index <= 7 and index >=1 :
                        completeRow = completeRow + cell.replace('\n', '').replace('\t', '') + '\t\t'
                data = data + completeRow + '\n'
        j = j+1
    return head + data

Here the bot scrapes the whole margins page for equity futures using requests and finds the desired table using bs4. Then a for loop parses every row of the table and a conditional checks if the second element of row (stonk name) matches the input ticker. It then goes on to return all the futures derivatives of the given ticker after cleaning the data for '\n\ & '\t' characters.

Here's the complete source for the python script.

Here is the list of terminal commands :

Enter 'python3 marginBoi.py <TICKER> <x>' where x can be :  
'com' : Commodity 
'eq' : Equities 
'eqfut' : Equity Futures 
'fxfut' : Currency Futures 
Example : 'python3 marginBoi.py BHEL eqfut'

I plan of adding more robust features to stonk-boi like maybe trade execution and stonk alerts.... Stay tuned !

As a tribute to this epic event I made a telegram bot to notify me about the total number of cases and the total number of deaths due to beer-flu at any give time - live. This was the first telegram bot I actually hosted on a cloud server. All of my previous bots were either self hosted on a SBC or used serverless functions to do the trick but this time I wanted to do it differently. I bought a cheapo $2/mo vps on 1&1 (cheapest vps plan I could find) pushed the script using SSH and ran it after detaching the terminal. All of this in under 30 minutes !

Here's some really poorly written code >

import logging
import telegram
from telegram.error import NetworkError, Unauthorized
from time import sleep
import requests
from bs4 import BeautifulSoup

update_id = None

countries = 'China\nItaly\nUSA\nSpain\nGermany\nIran\nFrance\nS. Korea\nSwitzerland\nUK\nNetherlands\nAustria\nBelgium\nNorway\nSweden\nCanada\nDenmark\nAustralia\nPortugal\nMalaysia\nBrazil\nJapan\nCzechia\nTurkey\nIsrael\nIreland\nDiamond Princess\nLuxembourg\nPakistan\nThailand\nChile\nPoland\nEcuador\nGreece\nFinland\nQatar\nIceland\nIndonesia\nSingapore\nSaudi Arabia\nSlovenia\nPhilippines\nRomania\nIndia\nPeru\nBahrain\nRussia\nEstonia\nEgypt\nHong Kong\nMexico\nPanama\nSouth Africa\nLebanon\nArgentina\nIraq\nColombia\nCroatia\nArmenia\nSerbia\nSlovakia\nKuwait\nBulgaria\nSan Marino\nTaiwan\nUAE\nAlgeria\nUruguay\nHungary\nLatvia\nCosta Rica\nDominican Republic\nLithuania\nJordan\nMorocco\nVietnam\nBosnia and Herzegovina\nFaeroe Islands\nAndorra\nNorth Macedonia\nCyprus\nBrunei \nMoldova\nSri Lanka\nAlbania\nBelarus\nMalta\nVenezuela\nNew Zealand\nBurkina Faso\nTunisia\nGuadeloupe\nSenegal\nGeorgia\nKazakhstan\nAzerbaijan\nCambodia\nPalestine\nOman\nTrinidad and Tobago\nUkraine\nRéunion\nUzbekistan\nCameroon\nMartinique\nLiechtenstein\nChannel Islands\nHonduras\nBangladesh\nAfghanistan\nDRC\nParaguay\nNigeria\nCuba\nGhana\nPuerto Rico\nJamaica\nMacao\nBolivia\nGuyana\nMonaco\nFrench Guiana\nGuatemala\nRwanda\nMontenegro\nTogo\nFrench Polynesia\nGuam\nMauritius\nBarbados\nIvory Coast\nKyrgyzstan\nMaldives\nMayotte\nGibraltar\nMongolia\nEthiopia\nAruba\nKenya\nSeychelles\nEquatorial Guinea\nTanzania\nU.S. Virgin Islands\nGabon\nSaint Martin\nSuriname\nBahamas\nNew Caledonia\nEswatini\nCayman Islands\nCuraçao\nCabo Verde\nCAR\nCongo\nEl Salvador\nLiberia\nMadagascar\nNamibia\nSt. Barth\nZimbabwe\nSudan\nAngola\nBenin\nBermuda\nBhutan\nFiji\nGreenland\nGuinea\nHaiti\nIsle of Man\nMauritania\nNicaragua\nSaint Lucia\nZambia\nNepal\nAntigua and Barbuda\nChad\nDjibouti\nEritrea\nGambia\nVatican City\nMontserrat\nNiger\nPapua New Guinea\nSt. Vincent Grenadines\nSint Maarten\nSomalia\nTimor-Leste\nUganda'

instructions = "Enter 'info' to get summary of the world.\nEnter 'country <country name>' for Country-Wise Info.\nEnter 'list' to get list of available countries."

def main():
    global update_id
    bot = telegram.Bot('TOKEN')

    try:
        update_id = bot.get_updates()[0].update_id
    except IndexError:
        update_id = None

    logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

    while True:
        try:
            echo(bot)
        except NetworkError:
            sleep(1)
        except Unauthorized:
            update_id += 1


def echo(bot):
    global update_id
    for update in bot.get_updates(offset=update_id, timeout=10):
        update_id = update.update_id + 1

        if update.message:
            if update.message.text.lower() == "info":
                update.message.reply_text(info())
            elif update.message.text.lower().split(" ",1)[0] == "country":
                update.message.reply_text(data(update.message.text.split(" ",1)[1]))
            elif update.message.text.lower() == "list":
                update.message.reply_text(countries)
            else:
                update.message.reply_text(instructions)

def data(country):
    i = 0
    page = requests.get("https://www.worldometers.info/coronavirus/")
    soup = BeautifulSoup(page.content, 'html.parser')
    table = soup.find('table')
    table_rows = table.find_all('tr')
    for tr in table_rows:
        td = tr.find_all('td')
        if i > 0:
            if td[0].text.lower() == country.lower():
                return td[0].text.strip() + " has " + td[1].text.strip() + ' total cases, ' + td[2].text.strip() + ' new cases, ' + td[3].text.strip() + ' total deaths, ' + td[4].text.strip() + ' new death(s), ' + td[5].text.strip() + ' total recoverd, ' + td[6].text.strip() + ' active cases, ' + td[7].text.strip() + ' serious critical cases.'
        elif i >= len(table_rows)+1:
            return "Invalid Country"
        i =i+1

def info():
    page = requests.get("https://www.worldometers.info/coronavirus/coronavirus-cases/")
    soup = BeautifulSoup(page.content, 'html.parser')
    i = soup.find_all('p')[0].get_text()
    return i


if __name__ == '__main__':
    main()

Here's what is does :

• Scrape the data from /www.worldometers.info and parse it using beautiful soup.

• Reply with this data every-time any message is sent to the bot.

• Its basically a zombie mutation of an echo bot.

I am planning to enhance its functionality by giving it access to country-wise data and rates available on worldmeters. UPDATE: Added the country-wise inquiring feature. Check it out here.

To try this bot use bot id > @CovidCaseReporter_bot on telegram. Enter 'info' to get summary and 'country <name>' to get specifics.

Stay Safe !