Day 21

Continue to writing the Flurry EDA report. It requires lots of data type formation and cleaning. And statistics knowledge is used to do analysis. It takes much more time than I expected due to my raw skill. Keep fighting! 

Total working hours:9.5

Day 18

I still can’t improve the query efficiency and decide to switch to next task:analyze Flurry. Flurry is a free analyze tool for mobile app. It basically analyses the major useful data that can evaluate an app’s performance. I am currently in the stage of finding important evaluation metric that can be reported. 

It is a very useful and practical skill that I can learn from this task. 

Day 17

Revise the dashboard with more efficient codes. But the dashboard load up time is still too long. Create index does not solve the problem. The processing job in R is actually quite fast. It turns out that the hard part is to write high efficient query. Damn!

Total working hours:9 

Day 14

My boss Chris came by the office today. He suggested that the data loading process from database to R was too slow when it had large volume of data. It took like 7mins to process 200M rows.

Chris pointed out that it cannot be avoided SQL if you want be a data scientist. When processing big data, query efficiently in SQL becomes a very important skill. So I guess I need to formally study SQL for the next few days. 

Chris is by far the most skillful and talented CS engineer I have ever seen. He almost can do any job without the need of using the mouse trackpad. He only uses keyboard command. No point and click of mouse. Damn!

Total working hours:6.6

Day 13

4th dashboard draft done. The rest of the work is about polishing layout and minor bug fix.

As a data scientists, I am asked to help with the IT problem just because I am the only “tech guy” in the office for now. I just can’t believe that the people seem like know nothing about computer today. Alas. 

Anyway, the overall experience is quite good. I hope I will still be part of this company when I study in HKUST. 

Total working hour:5.5

Day 12

Finally came back from my hometown. First day work in Shenzhen office. The chair and desk are not fully set up yet. It overall looks good. Meet new colleague. They are young and nice. 

The p2p van market in China is full of small competitors but doesn’t have a monopoly one yet. Our company is still in the stage of getting more drivers and engineers. So our service is not officially open to the public yet. Lots of things are going on and uncertainty remains. 

Just finished a analytics brief plan. The rough deadline is set. No more lazy work. 

Total working hour:4.5

Day 11

Finished draft 3 with a dashboard with operation and analytics option. Not bad.

I just found out there is a XTS package that can handle time series data so easy! OMG! I wasted like two whole days to write monthly, hourly, weekly, daily average, which can be done in a few lines of code. DAMN! I need to revise almost the whole program.

Better to read through those time series package shortly since dealing with the time format is a panic!

Total working hours:7