Weld logo
SQL TUTORIAL • LESSON 25, Tips & Tricks

LIMITing the amount of data with SQL

SQL can handle very large amounts of data. But if you’re working with very large data sets, this requires a lot of computing power, which can take up resources both in terms of time and money. So it’s a good idea to limit the amount of data you work with in SQL. Especially when you’re developing your models, it’s often enough to work with a smaller sample of the full data set, and then you can apply the final model on the full table. You can use the LIMIT function for this. The order line table contains 1000 rows. If you wanted to develop a model for this table by only experimenting with the first 5 rows, it could be done like this:

1select
2    *
3from
4    {{raw.e_commerce_sample.webshop_order_line}}
5limit
6    5

This returns this table:

A Table showing the LIMIT function

The order in which you place your ‘limit’ statement in your query is important. For example, if you want to add an ‘order by’ statement to the query above and order the query as:

  1. SELECT
  2. FROM
  3. LIMIT
  4. ORDER BY

The result would be to first select the 5 top rows in the raw table, and then order those 5 rows. If you change the order of the statements to:

  1. SELECT
  2. FROM
  3. ORDER BY
  4. LIMIT

The resulting table would instead be the 5 top rows after you’ve ordered them. You can try this on your own.

Next up

Formatting best practices

Go to lesson
Weld logo

Tired of scattered data? Sync data to your data warehouse with Weld in minutes with our powerful ETL, SQL Transformations, Reverse ETL and AI Assistant, Ed - connected to 100+ apps, files and databases.

Backed by leading investors
Frontline logoCherry logoInnnovation Fund logo
Twitter LogoLinkedIn Logo
GDPR logoSOC2
© 2024 Weld. All rights reserved.