I built SQL query executors for an open source database management system
[ posted 2024-05-12 @ 6:11 AM ]
Well, that's one way to learn SQL: by building the functions that actually operate on the data inside the memory pages of a database!
Building SQL query executors for an open-source DBMS (BusTub) has been an enlightening experience. This project, part of the CMU 15-445 graduate course "Database Systems," has challenged me to dive deep into the intricacies of SQL query processing and execution.
The main objective of this project was to implement query executors that can efficiently process a wide range of SQL queries, from simple SELECT statements to more complex JOIN operations. To achieve this, I leveraged the foundational knowledge gained from previous assignments (disk backed extendible hash table and the buffer pool manager). It was really exciting to see how SQL queries are executed under the hood.
Implementing SQL query executors also gave me a tangential understanding of SQL query parsing, query planning, and query optimization techniques, as the optimizations I implemented were done inside the optimizer component, which is a part of the query processing layer. I dove into multiple aspects of query execution, including building individual query executor nodes, accessing and manipulating data stored in the database and implementing optimizer rules to physically change the query plan to use more efficient nodes.
While this project didn't introduce new concepts in C++ development, it significantly enhanced my understanding of database engineering. Working on a large modern C++17 project like this has been invaluable. Over the course of the past three projects, I've acquired a wealth of knowledge in modern code style, move semantics, multithreading, concurrency, and more.
Overall, building SQL query executors has been a rewarding experience that has deepened my understanding of database systems and SQL query processing. Thanks for reading!