Replit released Replit Agent, an AI Software Engineer that helps build full-stack apps. Replit Agent includes a planner, a code editor, integrations to third-party APIs such as Discord and Stripe, and the ability to create and deploy Postgres databases powered by Neon.

In this tutorial, we’ll walk through how to use Replit Agent to build a news app that scrapes data from Hacker News, stores it in a Postgres database, and displays it using a Next.js front end. But first, let’s dive into what Replit Agent is and why it’s a powerful tool for developers.

Prefer watching over reading? You can watch the full tutorial on YouTube on how to create a scraper using Replit Agent, where we walk through the process of building this news app.

For those who enjoy a written guide, continue reading to learn how to use Replit Agent to build a fully functional news app that scrapes Hacker News, stores data in a Postgres database, and renders it using Next.js.

What is Replit Agent?

Replit Agent is an AI-powered development tool within the Replit ecosystem that allows you to easily create, debug, and prototype applications. By simply providing natural language prompts, you can instruct Replit Agent to generate fully functional apps, install dependencies, create back-end and front-end structures, and even deploy the final product.

You can think of Replit Agent as an intelligent coding assistant that helps you turn your ideas into reality quickly—whether it’s a stock data visualization app, a map of local landmarks, or, in our case, a news scraper app. It’s integrated with various databases like Postgres, and third-party applications such as Google Docs, Slack, Discord, and more.

Note:

It’s important to note that Replit Agent is currently in early access. This means the tool is still experimental, and while it does its best to fulfill requests, users may occasionally encounter errors or unexpected behavior. Replit is actively working on improvements and appreciates user feedback during this phase.

What Can You Do with Replit Agent?

Replit Agent can help you:

• Rapidly prototype apps with minimal coding
• Integrate with databases like Postgres
• Create both back-end and front-end code
• Debug and deploy applications
• Integrate external APIs and services such as Slack or Google Docs

Now, let’s go step by step to build our news app.

Step-by-Step Guide: Building a News App with Replit Agent

Step 1: Sign Up for Replit and Enable Replit Agent

1. Go to Replit and sign up for an account if you haven’t already.

2. Replit Agent is available in Replit Core and Replit Teams. Make sure you have access to one of these plans.

Step 2: Start a New Replit Agent Project

1. Once signed in, navigate to the Replit Agent interface.

2. Use a natural language prompt to instruct the agent to start building your news app. Here’s the prompt we used:

I want to build a news app that
1) continuously scapes news.yconbinator.com,
2) populate the data in a postgres database,
3) render the news in nextjs frontend

3. Click on “Start Building”. Replit Agent will then propose a step-by-step plan for the project.

Step 3: Approve the Plan

1. Review the proposed steps, which may include:

   – Implementing user authentication

   – Creating an API endpoint for scraping

   – Adding search functionality

   – Implementing caching

2. Approve the plan, and the agent will begin building the files and dependencies for your project.

Step 4: Watch the Project Build

As the agent starts working, you’ll see:

– Files being automatically generated (e.g., scraper, database models, API endpoints)

– Dependencies like Postgres and Next.js being installed

– A live console where the app is being compiled and executed

Step 5: Customize the Scraper

The scraper will continuously collect data from the Hacker News website. It’s built using:

– SQLAlchemy to manage database interactions

– Flask for handling API requests

You can inspect the scraper code, which will look something like this:

import requests
from bs4 import BeautifulSoup
from database import add_news_item

def scrape_hacker_news():
    url = 'https://news.ycombinator.com/'
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')
    
    items = soup.find_all('tr', class_='athing')
    
    for item in items:
        title_tag = item.find('a', class_='titlelink')
        if title_tag:
            title = title_tag.text
            url = title_tag['href']
            item_id = item['id']
            
            subtext = item.find_next_sibling('tr').find('td', class_='subtext')
            score = subtext.find('span', class_='score')
            score = int(score.text.split()[0]) if score else 0
            
            author = subtext.find('a', class_='hnuser')
            author = author.text if author else 'Unknown'
            
            add_news_item(item_id, title, url, author, score)

if __name__ == '__main__':
    scrape_hacker_news()

Step 6: Set Up the Postgres Database

The Replit Agent automatically integrates Postgres to store your scraped news data. 

You can manage and view the schema within the Replit interface:

1. Go to the Postgres section.

2. You’ll see tables like `news_items` that store information scraped from Hacker News, such as titles, URLs, and timestamps.

Step 7: Create the Next.js Front End

The front-end component of the news app will be generated using Next.js, allowing users to view the scraped news in a simple and responsive interface. The Replit Agent handles:

– Routing for different pages (e.g., Home, News Detail)

– Rendering the scraped news data

– Styling using CSS or a framework like Tailwind CSS

Step 8: Deploy the App

After ensuring the app works as expected, you can deploy it directly from Replit. Simply click the Deploy button, and Replit will handle hosting and deployment for you.

Step 9: Review the Results

You’ll now have a fully functioning news app with:

– A back-end that scrapes data from Hacker News in real-time

– A Postgres database that stores the scraped news items

– A front-end built with Next.js to display the news

Why Use Replit Agent for Building Apps?

– Speed: What would normally take days can now be done in minutes.

– Simplicity: The natural language interface allows even non-coders to build apps.

– Versatility: Integrations with third-party services like Postgres, Stripe, Slack, and more make it easy to expand your app’s functionality.

– End-to-End Development: Replit Agent handles everything from building and testing to deployment.

Final Thoughts

Using Replit Agent can significantly accelerate your development process, allowing you to quickly bring ideas to life with minimal hassle. Whether you’re building a quick prototype or a full-featured app, Replit Agent’s AI-driven capabilities make it easy and efficient.

Ready to give it a try? Head to Replit and start building your next project today!