Remember the talk I gave nearly a month ago about the innovative and efficient solution we invented at work for rendering dynamic forms? Here's the link in case you missed it (Sorry, it's in Greek):
🔗 talk
The main idea, is that we wanted to find a way to create dynamic forms and pages. We achieved it by utilizing what's called a json-schema. See here for more:
At some point during this talk, I'm explaining how we persist these distinct schemas that are being used by each page in our database. Up until now, we had an internal tool that used an SQL template and the metadata of each different page, and dynamically created the sql insertions/updates files. Recently, a new requirement came in from management which was to convert all this to XML files instead of SQL.
Broad overview of how the tool worked so far
I unfortunately cannot use the exact naming of the metadata, data structures or talk a lot about it in detail however, I'll do my best to give you a short explanation:
Let's say you have a page called Page A and it has it's own schema. Each page has two schemas, one for declaring all the fields and their types, and one schema for just changing the widgets/components that are being used for each field (This is being demonstrated in the youtube video above, by the way).
const all_schemas_of_all_the_pages_of_the_project = [
{
// Metadata
name: 'Page A',
schema: schema.ts,
uiSchema: uiSchema.ts,
}
]
As you can see, each element of this array, has its own name, schema and uiSchema which are simply very large typescript objects. Quick example:
// schema.ts
const schema = {
type: "object",
properties: {
fieldA: {
type: "string",
},
fieldB: {
type: "number",
},
}
}
export default schema;
The tool would take the all_schemas_of_all_the_pages_of_the_project array, perform some actions for each element:
- Firstly, it would convert these typescript objects to JSON files and organize them in a different output directory.
- Then, it would use its metadata and generate a single SQL file. The SQL file would first check if this Page A we just created, already exists in our database, or if it's a new one, and perform an insertion or update accordingly.
- Finally, it would copy and paste the newly generated SQL files to our migration repo, in order to run them in the next build and deployment.
The New Requirement
The most challenging aspect of creating the tool was figuring out how to automatically generate SQL files for all our pages while ensuring that there was no risk of anything breaking during insertion or updating by basically respecting the database schemas.
As you may understand, the new requirement meant that the most difficult part of the tool had to change.
I spent many hours initially recalling how Liquibase works, then understanding how XML files are utilized by Liquibase, and creating a few simple examples. I needed to ensure a solid understanding of basic XML tags, attributes, and so on. So, I dedicated my initial hours to studying and analyzing. Once I got the hang of the fundamentals, I transitioned to coding the new XML template for our tools.
There were a lot of cases for me to cover and make sure they were handled, due to our database schemas and all of the contraint keys we had. So what I did was to write down all the possible scenarios, grouping them by the most easiest one to handle, up to the hardest one.
I started coding the template and felt a real sense of accomplishment. Progress was smooth, and the initial test cases worked without any problems. I was happy with the results, not because it felt easy or anything, but because I knew that the last cases would be incredibly complex and hard, and I would definitely need more time (which wasn't a great asset at the time, due to the pressure of the deadline).
Pushing for Different Reasons
This refactor was probably one of the most difficult challenges I have faced in my career so far. However, I knew that I had to make this work for different reasons.
The tool sees daily use without exception, ensuring our development environment stays up-to-date with the latest schemas for each page. Spending over an hour each day on updating or creating new schemas for new pages would be highly time-consuming and inefficient. It would impact all project teams, causing varying degrees of inconvenience.
In addition, this tool is my creation, and for that reason, it should function flawlessly. If it doesn't, I take it more personally because I am its owner. Being something I invented, it needs to operate without any issues. Failing to achieve this refactoring would signal the end of this internal tool, as it would be deprecated and I wouldn't let that happen.
Another motivation for making this work was to prove once again that I can go above and beyond. I aimed to demonstrate my ability to face any challenge, learn, adapt, and make things happen.
The New Manager
A few weeks back, a new manager joined our project, and the management assigned this task to him. When he inquired with his new team about who could assist him the most, they pointed to Elijah because I had already built the first version of this tool for generating the SQL files.
I felt incredibly honored to have my name brought up!
We worked together, he helped me a lot with testing more and more scenarios and made sure we wouldn't face any surprises. He is a very experienced individual with a lot of respect and kindness. We worked together like I've never worked with anyone before, and we managed to achieve such a huge milestone in a very short timeframe.
What stood out to me was his consistent acknowledgment of my efforts. He made a point to mention my name at every opportunity, ensuring I received credit for the hard work I put in. His message in our project's chat was:
Lastly Special thanks to Elijah for his valuable contribution, commitment and effort ... making happen in such a short timeframe a such significant change.
As a somewhat new engineer, these small gestures fuel my passion for what I do. I don't take anything for granted.
We made it
In the end, we did it! Refactoring our internal tool to generate XML files was tough, but we pulled through. Refactoring our code took some serious effort, but it was worth it. The new manager was a big help, and together we smashed our deadline. Getting recognized for my work felt awesome, and it's moments like these that keep me going. This experience taught me a lot and made me even more excited about being a software engineer.
If you're reading this, I really appreciate you. Thank you! :)