In order to reduce the amount of data parsed by query, it's important to organize your S3 bucket in separated folder. When you are using time based log files, it makes sense to create partition based on folder created by your log system, for example the ALB/ELB log : Year, Month, Day. Obviously, you can create partation according your own project structure. First create a partitioned Table Query Here Then Add your Partition when needed (through Lambda)