The 8MC Software Engineering has been finished. It has been a great project where I have learnt a lot and worked with wonderful people (well, one of few projects where the issue of freeloaders did not come up). 

Although software engineering was not new to me. That’s a thing I can do best when it comes to computing fields. However, I have learnt a whole lot of new things: profiling, unit testings, design critique,…

When I first started, I thought it would be all about coding and UML diagrams. It ended up not. Little coding, no UML diagrams, but a whole lot of report writings, design & code review, refactoring, profiling and testing and testing and testing… It is different from other SE courses. And I believe this credit comes to the teaching staffs. A project leader, as a lecturer, teaches you SE differently from what a researcher does. 

In the last lecture, everyone of us was given 30 sec to say something, excluding thanking the teaching staffs. A whole bunch said “I have nothing to say”. Typical software engineers haha. Anyway, this is what I planned to say is:

“Remember when you shared with us that because of “The Mythical Manmonth” by Fred Brooks, you realized that SE is not just about coding and you decided to become a software engineer, worked in the industry and now teaching software engineering? It was a sincere moment, and Caleb, who was sitting next to me told me “I think he’s crying”. It was touching to me to see someone who does what s/he likes, and enjoy it. And I believe a person will do a good job in that situation. In this case, it is true. I will not say thank you. But Good Job!”

But when I mentioned the moment, he looked so uneasy that I forgot to repeat what Caleb said. Anyway, it is always great to tell the person that s/he has done a good job :”)

OK, a short summary of some interesting things I have done in this project:

1. Implement table joining

We have  a query language which allows “AND” operation between clauses. So we need to join the results of clauses together during evaluation. I ended up implementing this table “column based” and all things such that filtering, joining and sorting.

A problem comes up when we need to decide whether to join results right after evaluating each clauses or not. If you do and if the result of the joining is empty, you can terminate the evaluation early. But there is also a risk of expanding of results (joining can result in a Cartesian multiplication). I have implemented a decision making part for it. It works much better with big results set now.

However, the last improvement was too close to the deadline. The profiling report shows that it was slower than the previous version. No time to investigate the problem. And I’m a bit freaked out about the fact that it could have bugs. This is the last point of the evaluation. If it’s buggy, it will screw up everything. [praying]

2. Sorting a table

Same table above. We need to prioritized sort the column. First column has the highest priority and so on. It took me a lot of time. Sometimes I thought I found a smart solution (ToString() all the rows and sort strings) but it did not work as we wanted. So well, I did something like Radix Sort for this: use a stable sorting to sort from the last row to the first row. And it worked. However, I used C# CompareTo() default to compare 2 strings and it uses strings’ lengths as the first characteristic to sort so “A” will come up before “aaa”. Anyway, it’s an assumption for the product that we can make, so I just leave it like that.

3. Undeterministic unit testing

Well I appreciate testing doesn’t mean that I’m happy to do testing. Especially uniting testing. So I was creating tests for the optimizer of our system. This optimizer rearranges the order of the clauses we will evaluate so that it’s most likely that we can evaluate it faster. Just a static optimizer, nothing fancy about it. Anyway, I need to test if the order of clauses it produces is the order I want. Create a multiple lists of clauses with different order of adding clauses into the list and test them all? I’m too lazy to do so. So I did something like a randomized comparer that will return a random value when you ask it to compare two objects. I use that comparer to sort my input list before I give it to the optimizer. The output list should always be the same.

And I wonder if there is any order of the input list that would make my optimizer fail… So far I haven’t found that in my runs muahahahaha and I don’t know how I can reproduce that order if I ever find my test case fails muahahahahah…