About Myself
I’m Liu Junfeng, currently working in PingCAP as an infra engineer. Recently interested in:
- Streaming system
- Database system
- Cloud native system
- Robust and comprehensible system design
My Work
Working in PingCAP, I’m mostly focusing on building data migration system, capable of both full migration and incremental migration (or, so-called Change Data Capture system). In terms of the full migration, the project I’m in is Lightning, which can import data from files to TiDB. The most distinctive merit of using Lightning is that it is able to decode and encode data as key-value pairs and directly ingest them to TiKV, the KV storage engine of TiDB, which is exponentially faster than simply executing SQL via TiDB.
Another project I’m participating is TiFlow. It’s a high-performance data migration platform being able to capture change-data from MySQL-like databases, such as MySQL, MariaDB, Aurora, TiDB, etc, to other data storage systems, including TiDB, Kafka, MySQL, and etc. I’m responsible to build the migration system from MySQL to TiDB (often known as DM). In addition to data migration, we have built a lot of interesting and helpful features to make the system highly reliable, available and usable.
The two systems are open-source. Please feel free to reach out to us by filing issues or contacting me by e-mail if you have any questions or precious suggestions.
Besides, I’m recently taking part in developing the TiDB Cloud and supporting import service on it. But I don’t have much knowledge about cloud computing and still work hard to comprehend it.
I’m currently looking for a PhD opportunity researching software system, database system, or data processing system. Please contact me if you are interested in my work.
If you are willing to take a close glimpse of my work:
Contact
liujunf9@gmail.com