My Simple Approach to AI Agents Doing Useful Work

Hey everyone, Jake here from clawgo.net, and wow, what a week it’s been. My inbox has been absolutely swamped with questions about AI agents, specifically how to actually get one doing something useful without needing a PhD in computer science. It seems like every other day there’s a new framework, a new model, or some new buzzword promising the moon, but for us regular folks, it’s often hard to see the forest for the trees.

So, today, I want to tackle something incredibly practical: building a simple, self-improving AI agent to manage your daily digital detritus. We’re talking about the kind of stuff that eats up an hour or two every week – sorting downloads, organizing screenshots, maybe even summarizing those ridiculously long email threads you keep meaning to read. My goal? To show you how to set up an agent that learns from your actions, slowly but surely, and makes your digital life just a little bit smoother. Think of it as your own personal digital butler, but one that gets smarter the more you interact with it.

We’re not building Skynet here, folks. We’re building a humble, helpful assistant. And we’re going to do it using a combination of Python, some basic file system monitoring, and a sprinkle of a relatively new concept called “reinforcement learning from human feedback” (RLHF), but applied in a super simplified, local way. No fancy cloud GPUs needed, just your trusty laptop and a willingness to experiment.

The Problem: Digital Clutter is a Time Sink

Let’s be honest. How many of you have a “Downloads” folder that looks like a digital warzone? Or a desktop littered with screenshots named “Screenshot 2026-04-17 at 10.34.23 AM.png”? Yeah, me too. For years, I just accepted it as part of the digital grind. I’d spend 15-20 minutes every few days just moving files around, deleting duplicates, and generally trying to impose some order on the chaos. It’s not hard work, but it’s tedious, repetitive, and frankly, a waste of my limited brainpower.

I tried various automation tools over the years – Hazel on Mac, some custom PowerShell scripts on Windows. They helped, sure, but they were rigid. They did exactly what I told them to do, no more, no less. If my habits changed, or if a new type of file started appearing, I had to manually update the rules. That’s not an agent; that’s just a fancy macro.

My “aha!” moment came a few months ago when I was messing around with some open-source LLMs and thought, “What if I could build something that watches what I do, and then tries to replicate it, getting better over time?” That’s the core idea behind our self-improving agent.

Our Agent’s Mission: The Auto-Sorter

For this project, we’re going to build an agent I’ve affectionately dubbed “Auto-Sorter.” Its primary mission is to watch a specific folder (let’s say, your Downloads folder), and when a new file appears, it will suggest where that file should go. The magic happens when you provide feedback: if its suggestion is good, you approve it; if not, you tell it where it should have gone. Over time, Auto-Sorter learns your preferences.

Think about it: you download an invoice. Auto-Sorter suggests moving it to Documents/Invoices/2026. You agree. Next time, it’s more confident. You download a Python script. It suggests Development/Scripts/Python. You disagree and move it to Projects/Current Project/Code. Auto-Sorter remembers that. It’s not just following rules; it’s learning your intent.

The Core Components: How We’ll Build It

We’ll need a few pieces to make this work:

1. File Watcher: Something to detect new files. Python’s watchdog library is perfect for this.
2. Decision Engine: The “brain” of our agent. For simplicity, we’ll start with a basic rule-based system, but the crucial part is how it updates these rules. We’ll use a simple JSON file to store our agent’s “knowledge.”
3. Feedback Mechanism: A simple way for you to tell the agent if it did well or not. A command-line prompt will suffice for now.
4. Learning Loop: The part that modifies the decision engine’s knowledge based on your feedback. This is where the “self-improving” aspect comes in.

Let’s get our hands dirty with some code. First, make sure you have Python installed, and then install watchdog:

pip install watchdog

Step 1: The File Watcher

Here’s a basic setup for watching a folder. We’ll put this into a Python script named auto_sorter.py.

import time
import os
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

# Configuration
WATCH_FOLDER = '/Users/jake/Downloads' # CHANGE THIS TO YOUR DOWNLOADS FOLDER!
KNOWLEDGE_FILE = 'sorter_knowledge.json'

class MyHandler(FileSystemEventHandler):
 def on_created(self, event):
 if not event.is_directory:
 print(f"New file detected: {event.src_path}")
 # Here's where we'll call our agent's decision-making logic
 process_new_file(event.src_path)

def process_new_file(file_path):
 # This is a placeholder for our agent's logic
 print(f"Agent needs to decide what to do with: {file_path}")
 # In a later step, this will return a suggested path

if __name__ == "__main__":
 event_handler = MyHandler()
 observer = Observer()
 observer.schedule(event_handler, WATCH_FOLDER, recursive=True)
 observer.start()
 print(f"Auto-Sorter is now watching: {WATCH_FOLDER}")
 try:
 while True:
 time.sleep(1)
 except KeyboardInterrupt:
 observer.stop()
 observer.join()
 print("Auto-Sorter stopped.")

Run this script, and then drop a file into your Downloads folder. You should see “New file detected: [path to file]” appear in your terminal. Success! Our agent has eyes.

Step 2: The Decision Engine and Knowledge Base

Now for the brain. We’ll store our agent’s “knowledge” in a simple JSON file. This file will contain mappings of keywords/file types to destination folders. For instance, if a file contains “invoice” in its name or is a PDF, it might suggest the “Invoices” folder.

Let’s create a sorter_knowledge.json file:

{
 "rules": [
 {"pattern": "invoice", "type": "keyword", "destination": "Documents/Invoices"},
 {"pattern": ".pdf", "type": "extension", "destination": "Documents/PDFs"},
 {"pattern": "screenshot", "type": "keyword", "destination": "Pictures/Screenshots"},
 {"pattern": ".jpg", "type": "extension", "destination": "Pictures/Images"},
 {"pattern": ".png", "type": "extension", "destination": "Pictures/Images"},
 {"pattern": ".py", "type": "extension", "destination": "Development/Python_Scripts"}
 ],
 "feedback_history": []
}

Now, let’s modify our process_new_file function to use this knowledge. We’ll also add a function to load/save our knowledge.

import json
# ... (other imports) ...

# ... (WATCH_FOLDER, KNOWLEDGE_FILE) ...

def load_knowledge():
 if os.path.exists(KNOWLEDGE_FILE):
 with open(KNOWLEDGE_FILE, 'r') as f:
 return json.load(f)
 return {"rules": [], "feedback_history": []}

def save_knowledge(knowledge):
 with open(KNOWLEDGE_FILE, 'w') as f:
 json.dump(knowledge, f, indent=4)

def suggest_destination(file_path, knowledge):
 file_name = os.path.basename(file_path).lower()
 file_extension = os.path.splitext(file_name)[1]

 # Simple scoring system for now
 best_match_destination = None
 max_score = 0

 for rule in knowledge['rules']:
 current_score = 0
 if rule['type'] == 'keyword' and rule['pattern'] in file_name:
 current_score += 2 # Keyword match is strong
 elif rule['type'] == 'extension' and rule['pattern'] == file_extension:
 current_score += 1 # Extension match is good

 if current_score > max_score:
 max_score = current_score
 best_match_destination = rule['destination']
 elif current_score == max_score and best_match_destination: # If same score, prioritize existing rule
 # Could add more complex tie-breaking, but for now, first match wins
 pass

 # Default to a "Misc" folder if no strong match
 if not best_match_destination:
 return os.path.join(WATCH_FOLDER, "Misc")

 return os.path.join(WATCH_FOLDER, best_match_destination) # Ensure path is absolute for safety

def process_new_file(file_path):
 knowledge = load_knowledge()
 suggested_path = suggest_destination(file_path, knowledge)

 print(f"\nAuto-Sorter suggests moving '{os.path.basename(file_path)}' to: {suggested_path}")
 # ... (Feedback mechanism will go here) ...

Now, when you drop a file, it will suggest a destination based on your initial rules. Try it with a .pdf or a file with “invoice” in its name.

Step 3: The Feedback Mechanism and Learning Loop

This is where our agent truly becomes “self-improving.” After it makes a suggestion, we’ll ask for your input. If you confirm its suggestion, great! If you provide a different path, we’ll use that to update its knowledge.

# ... (previous code) ...

def process_new_file(file_path):
 knowledge = load_knowledge()
 suggested_path = suggest_destination(file_path, knowledge)
 original_filename = os.path.basename(file_path)

 print(f"\nAuto-Sorter suggests moving '{original_filename}' to: {suggested_path}")
 user_input = input("Accept (y/n) or provide new path: ").strip().lower()

 final_destination = suggested_path
 if user_input == 'y':
 print("Accepted. Moving file...")
 # Actually move the file
 try:
 os.makedirs(suggested_path, exist_ok=True)
 os.rename(file_path, os.path.join(suggested_path, original_filename))
 print(f"Moved '{original_filename}' to '{suggested_path}'")
 record_feedback(file_path, suggested_path, True, knowledge)
 except OSError as e:
 print(f"Error moving file: {e}")
 elif user_input == 'n':
 new_path_input = input("Enter the correct destination folder (e.g., 'Documents/Reports'): ").strip()
 if new_path_input:
 final_destination = os.path.join(WATCH_FOLDER, new_path_input)
 print(f"User provided new path. Moving file to: {final_destination}")
 try:
 os.makedirs(final_destination, exist_ok=True)
 os.rename(file_path, os.path.join(final_destination, original_filename))
 print(f"Moved '{original_filename}' to '{final_destination}'")
 record_feedback(file_path, final_destination, False, knowledge)
 except OSError as e:
 print(f"Error moving file: {e}")
 else:
 print("No new path provided. File remains in place.")
 record_feedback(file_path, None, False, knowledge) # Record non-action
 else:
 print("Invalid input. File remains in place.")
 record_feedback(file_path, None, False, knowledge) # Record non-action

 save_knowledge(knowledge) # Save updated knowledge after each action

def record_feedback(original_file_path, actual_destination, accepted_suggestion, knowledge):
 file_name = os.path.basename(original_file_path)
 file_extension = os.path.splitext(file_name)[1]
 
 # Store enough info to potentially learn from
 feedback_entry = {
 "file_name": file_name,
 "extension": file_extension,
 "suggested_destination": suggest_destination(original_file_path, knowledge), # The agent's original guess
 "actual_destination": actual_destination,
 "accepted_suggestion": accepted_suggestion,
 "timestamp": time.time()
 }
 knowledge['feedback_history'].append(feedback_entry)
 
 # Simple learning: If a suggestion was rejected, and a new path provided,
 # create a new rule or strengthen an existing one.
 if not accepted_suggestion and actual_destination:
 # Check if a similar rule already exists
 found_rule = False
 for rule in knowledge['rules']:
 # For simplicity, if the exact extension or a keyword in the file name
 # already points to this destination, we consider it.
 if (rule['type'] == 'extension' and rule['pattern'] == file_extension and 
 os.path.join(WATCH_FOLDER, rule['destination']) == actual_destination):
 # We could add a "weight" or "confidence" to rules here
 found_rule = True
 break
 if (rule['type'] == 'keyword' and any(kw in file_name.lower() for kw in rule['pattern'].split(',')) and
 os.path.join(WATCH_FOLDER, rule['destination']) == actual_destination):
 found_rule = True
 break

 if not found_rule:
 # Let's try to infer a rule.
 # If the file has a specific extension, prioritize that.
 # Otherwise, try to extract a keyword from the filename.
 
 # Option 1: Rule based on extension
 if file_extension and actual_destination:
 new_rule = {
 "pattern": file_extension,
 "type": "extension",
 "destination": os.path.relpath(actual_destination, WATCH_FOLDER) # Store relative path
 }
 if new_rule not in knowledge['rules']: # Avoid exact duplicates
 knowledge['rules'].append(new_rule)
 print(f"Agent learned new extension rule: {file_extension} -> {new_rule['destination']}")

 # Option 2: Rule based on keywords (more complex, but powerful)
 # For this simple example, we'll stick to basic extension/filename keywords.
 # A more advanced agent might use an LLM to extract relevant keywords.
 # Here, we'll just check if the filename *itself* could be a keyword.
 common_words_to_ignore = ['the', 'and', 'a', 'of', 'for', 'to', 'from']
 potential_keywords = [
 word for word in file_name.lower().replace(file_extension, '').replace('.', ' ').split() 
 if len(word) > 3 and word not in common_words_to_ignore
 ]
 if potential_keywords and actual_destination:
 # Let's just pick the first relevant looking word for now
 keyword_to_use = potential_keywords[0] 
 new_rule = {
 "pattern": keyword_to_use,
 "type": "keyword",
 "destination": os.path.relpath(actual_destination, WATCH_FOLDER)
 }
 if new_rule not in knowledge['rules']:
 knowledge['rules'].append(new_rule)
 print(f"Agent learned new keyword rule: '{keyword_to_use}' -> {new_rule['destination']}")

 # A more sophisticated learning algorithm would re-evaluate all rules,
 # perhaps using a simple Bayesian update or a shallow neural network
 # based on the entire feedback history. For now, we're doing direct rule creation.

Now, run the script, drop a new file, and interact with it. If you move a .docx file to a new Documents/Reports folder, and then drop another .docx, the agent should suggest Documents/Reports next time!

Refinement: Making it Smarter (A Glimpse into the Future)

Our current “learning” is pretty basic: if you correct it, it adds a new rule. This is a good start, but a truly smart agent would do more:

• Rule Weighting: Instead of just adding rules, each rule could have a “confidence” score. Correct suggestions increase confidence; incorrect ones decrease it. The agent then picks the rule with the highest confidence.
• Contextual Learning: What if the file came from a specific website? Or was downloaded at a specific time of day? These could be additional data points for learning.
• Natural Language Processing (NLP): Instead of just keywords, a small, local LLM could be used to understand the content of text files (if applicable) or to extract more meaningful keywords from filenames.
• Proactive Suggestions: Instead of waiting for you to move a file, it could periodically scan for unorganized files and make suggestions.

For example, to implement rule weighting, our sorter_knowledge.json might look like this:

{
 "rules": [
 {"pattern": "invoice", "type": "keyword", "destination": "Documents/Invoices", "confidence": 0.8},
 {"pattern": ".pdf", "type": "extension", "destination": "Documents/PDFs", "confidence": 0.0},
 {"pattern": ".docx", "type": "extension", "destination": "Documents/Reports", "confidence": 1.0}
 ],
 "feedback_history": [...]
}

And our suggest_destination function would pick the rule with the highest confidence score, while record_feedback would update these scores. If a rule leads to a correct suggestion, its confidence goes up. If it leads to a rejection, its confidence goes down. This is the essence of simple reinforcement learning from human feedback.

Actionable Takeaways

1. Start Small & Simple: Don’t try to build an all-encompassing AI butler on day one. Pick a single, annoying, repetitive task. My “Auto-Sorter” is a perfect example of a manageable scope.
2. Define Your Feedback Loop Early: The “self-improving” part of an agent relies entirely on feedback. How will you tell your agent it did well? How will you correct it? Make this as easy and frictionless as possible. For our agent, a simple ‘y/n’ or a new path is enough.
3. Iterate, Don’t Perfect: Your first version will be clunky. It will make mistakes. That’s the point! Every mistake you correct is a learning opportunity for your agent.
4. Local First: You don’t need expensive cloud services to build useful agents. Many tasks can be handled right on your machine, keeping your data private and your costs zero.
5. Think About Data Representation: How will your agent store its “knowledge”? A JSON file, a simple database, or even a plain text file can work. The key is that it’s structured in a way that your agent can easily read and update.

Building an AI agent isn’t about magical, sentient beings (yet!). It’s about automating tasks, learning from interaction, and making your digital life just a little bit easier. This Auto-Sorter project is a basic starting point, but it illustrates the fundamental principles. With a bit more Python, some clever heuristics, and your consistent feedback, you can evolve this into a surprisingly powerful personal assistant.

Give it a try! Modify the code, break it, fix it, and watch your own little AI agent grow smarter. Let me know in the comments what kind of agents you’re dreaming up or what challenges you’re facing. Happy coding!

🕒 Published: April 16, 2026