Before you begin this tutorial, make sure you've completed the Quick Start Guide
In this tutorial, we'll automate a simple process: sending home listings from Redfin to a Google Sheet for comparison shopping.
We'll use this Redfin page to develop our workflow. There is a video at the end of of this tutorial demonstrating what’s covered in the text.
0. Set Up your Google Sheet
- Create a new Google Sheet and give it a name, such as "Redfin Data"
- In row 1 create the following column headers
- Address
- Price
- Beds
- Baths
- Square Feet
- My Rating
1. Place a Button
Open the Page Editor
Start by navigating your browser to a property listing: Redfin. Then, open the PixieBrix Page Editor.
Need help opening the Page Editor? Follow the steps here:
Grant Access
The first time you open the Page Editor on a new webpage, you need to grant PixieBrix access to the page. You can grant permanent access by either:
- Clicking Grant Permanent Access, or
- Granting temporary access by clicking on the PixieBrix extension in the Chrome Extensions dropdown and then refreshing the page
If you had the Page Editor open before navigating to the Elea page, you'll need to close the Page Editor by clicking the X in the top right corner, and then re-open it to provide access.
Place a Button
- Click Add in the top left of the Page Editor and choose Button
- Then hover your cursor over either of the Redfin buttons in the button group in the top right of the page, which includes
[Log In, Sign Up]
- Click and PixieBrix will add a new custom button "Action" at the end of the button group. I'm choosing to add a button up here because I know it will be on every listing page
Customize the Caption
To customize the Caption, replace the word "Action" with "Google Sheets." You'll see the button on the page update with your new caption
The button should now look like this
2A. Read Data
To select data from the page (even data not directly visible in the interface), we'll add a Component Reader brick. Click the "Add Button"
In the Brick Selection modal, click "Component Reader".
When you add a brick, the Page Editor automatically selects that brick for configuration
Identify the Framework (Automatic)
The Component Reader automatically detects what front-end framework the site creator used to create the site
For Redfin, it will detect the popular React framework:
Select the Page Element
- Start by clicking the Data tab in the Page Editor
- PixieBrix should detect React as the website's Framework. If React is not detected, select it from the Framework dropdown
If PixieBrix did not automatically detect "React - 16.13.1", choose it in the Framework dropdown before continuing
Select the Page Element
Next, we'll tell PixieBrix where to read the data from on the page
Click the pointer icon for the Component Selector field to toggle selection mode
Hover your mouse over the section of the page shown below. When the purple overlay looks like thee following, click to select the page element:
Find the Property Path to Data
Let's double-check what data PixieBrix found. In the Data Panel to the right of the field, switch the "Preview"
As you can see, PixieBrix found all sorts of data
Address:
Click the page icon next to streetAddress to copy its property path to your clipboard: @data.streetAddress
You can see the following property paths for address:
@data.streetAddress@data.city@data.zip@data.state
Price:
Next, search for "1874" in the Page Editor Search field, and you should see the following results once you expand priceLabel:
Therefore, property path for the price is: @data.priceLabel.props.definition.props.predictedValue
Beds:
Next, search "beds" in the Page Editor Search field, and you should see the image below.
The property path is @data.beds
Baths:
Next, search "baths" and you'll find the property path: @data.baths
Square Feet:
Search for "1,826" and you'll find the property path: @data.sqFt
You'll need these property paths to configure the Google Sheet Effect in the next section of the tutorial. You can click the 📃 icon next to each property path to copy it to your clipboard.
2B. Annotate Data (Optional)
We added a “My Rating” field to our Google Sheet so you could add a personal rating to a given listing. In this step, we’ll configure a configure a popup modal so you can add a personal rating. Feel free to skip this step if you don’t want to annotate the data.
Add a Popup Modal
Click “Add” and search for “show a modal” and then + Add brick
Now when you click your “Google Sheets” button, a popup form appears over the page
Configure the Popup Modal
- Type “My Rating” in the “Form Title” field
- Scroll down to Current Field and use “myPersonalRating” for the Name
- Change the Label to “My Personal Rating”
- Change Field Description to “My personal rating from 1 to 10”
- Change Input Type to Dropdown and add ten Options for ratings 1-10
Note that the property path to this data will be @form.myPersonalRating
Now when you click your “Google Sheets” button, your newly configured popup modal appears over the page
3. Send to Google Sheet
Configure the Google Sheet
To select data from the page (even data not directly visible in the interface), we'll add a Add Google sheet row brick. Click the "Add Button" and search "add google" and click "Add brick"
- Under Google Sheet click Select
- Choose the spreadsheet you created for this tutorial, Redfin Data, and click Select
- For tab name, choose the sheet you want to send the data to. In our case thats "Sheet 1" because we didn't rename the sheet
- Add each property path to its corresponding row in the spreadsheet
Note that when there is more than one variable, you'll need to surround the property path with two sets of braces {{
Test your Workflow
Press the Google Sheets button to scrape the data you selected. Try it from different listing to make sure it works. Once you're happy with the button, click Save to scrape more listings in the future.
Under construction: this video is currently being updated for PixieBrix release 1.5.0
