Data Engineering 101 - 7 ideas how to use Your Own data

When we think about data, gathering or streaming, analyzing and visualizing. In most of the cases we have “big” things in minds. Companies that process terabytes of data and make decisions based on them. However there is also another side of it. Private one, that all of us are able to leverage. We are all generating tons of data daily, location data, social media, sport activities, spendings etc.

Most of them are available through various applications, that are providing (some basic / some more advanced) analytics.

This articles describes 7 ideas how to use your own data, and is part of a bigger cycle – Data Engineering 101.

1. Habits tracking and monitoring

Keeping track of your habits is a great point to start this list. Especially, as we’ve just started 2019. Most of us have some sort of habits (hopefully only good ones :)) for instance: daily training, meditation, reading etc.

There are hundreds of ways to track them. Ranging from various mobile applications, through web applications, ending on plain pen and paper.

Even more reasons exists, as to why we are tracking them. What is common I assume is this great feeling of accomplishment once all tasks from our lists are “checked” and done. Moreover when this would be X day in a row – you know that feeling right 😉

Why not make use of this data, and instead of just checking when and whether it was done, push it a bit further. What was the reason of not doing a particular habit, was it correlated with something that has happened at that particular day. Example: Waking up early (~ 5:00 AM) was missed, due to tough training in previous day, or a good party.

2. Evaluating your training records (sports)

For those of you that are running, biking, swimming, doing crossfit, more than once per month. I would expect that you are using sport apps (Endomondo, Nike+ etc.), to track your progress, right? Some of these applications are great out of the box. Some have really cool features, but for additional price.

You can check weather conditions, calories burnt record, how your friends are doing, all this stuff in the glimpse of an eye.

But, are you checking your overall progress or just analytics for this certain training? Is it beneficial for you that you’ve ran faster, than some random person in your age group. Maybe better for You would be to compare what is the progress versus previous year (if you were gathering such data or using same app)?

Tracking trainings is one thing, getting insights based on them, measuring progress, evaluating the causes of good vs bad training is completely other level.

3. Better track of your financial records (income / spending monitoring)

As Master Yoda from Star Wars used to say “Do or do not, there is no try”, same is with this point. You are either tracking your expenses or you are not. Even with all cool banking features, that are sort of doing this for us (classifying your spending,when you are paying using your bank account), it all comes down to one simple sentence. Are you regularly checking on what you are spending money 🙂

Honestly, I’m doing that for couple of years now, this helps me in setting up a monthly budget, knowing more or less how much I spend on clothing, cosmetics, leasure, food.

In some strange way, this “habit” gives me a confidence and a sense of feeling that I’m in control of my finance world. Of course it would be great no to write everything down manually (in Excel or Google Sheet or Other App) but still I do like it and it is worth a hussle.

Thanks to having such data, I’m able to exactly say what is my month over month spending rate in food category for instance (or any other calendar period). In what months I will need more money, based on previous years etc.

This can also give you some guidance that can be used with data from point 1. In my case I’ve found out that I’m spending quite a lot (~ 12 PLN per week) in my office vending machine. Which is not something big, but still when you multiply it by 4 weeks and 12 months perspective is different. In addition, you understand what exactly you were spending that on, hmm. You may realize, that this money could spent on other stuff, and definitely on more healthy products 🙂

4. Car Profitability Analysis

In this example I’m using car, as it is easier for me to give you the root idea behind. In general you can add in place of a car, any item that you are an owner off, and it generates some sort of expenses. Question is, is it really worth the money?

Yes, yes, we could end this point with a “famous” it depends. It always depends on many factors, and definitely not only money. However in this particular case, I’m taking under considerations, only expenses around a car, I’m not counting, time saved on trips, comfort etc. (stuff that I’m not able to really measure in money).

To the point, when owning a car you need to pay for its service, parts, gasoline, insurance and others. Now compare that with amount of miles / kilometers that you’ve driven since buying it. Based on that you can say how much you are spending per mile/kilometer on your car. Is it comparable to public transport, taxi, uber?

If you are living in a big city like myself, with fairly good public transport, and you are able to survive in a bit longer rides to your office / school and back. Than maybe, just maybe it would be a point to consider, in favor for a change.

5. Gasoline price changes compared to places where you fill the tank.

This one comes naturally after point 4 and 5. If you are monitoring your expenses and checking your car spending records. You may end up with a big amount of data connecting with gasoline prices, compared with places where you re-fuel.

You may see how the price was changing, in previous months. When it was the highest and when lowest. What may be the reasons behind it? Why on a Station A, gasoline is around 10% cheaper than on Station B, is it worse there, or owner has lower margins?

How frequently you need to re-fuel (if you are filling the tank once it is empty), once per month, twice or more?

When is the best time, to go to the gas station (and of course obvious answer is when my tank is nearly empty), can I plan that?

6. Monitoring progress of your blog or personal webpage, social media profiles

Please don’t get me wrong, you don’t need to monitor it all by yourself. Nearly all social media platforms are giving you insights about number of visitors, “likes” amount, number of people that have seen your tweet. In addition there is whole Google Analytics at your service, plus a lot of other products that are monitoring and analyzing activity and mentions of your “brand” in the internet.

What I’m trying to tell you that having this data is not enough, make use of it. Correlate it with other (maybe) external factors.

Think why one blog post had more visitors than the other based on Google Analytics. Where you more active in social media when the first was published. Was your topic, trending in internet. Are people more interested in such information at this particular season than your other topics?

Monitoring, gathering, analyzing and getting insights this is your key to success.

7. Analysis of your marketing activities on worth of brand / products

This point can be a subpart for number 6, but I’ve splitted it on purpose. When you are building your brand – sharing knowledge, giving some valuable insights to others etc. – it would be worth considering what impact you are really making. One of the ways is to measure how big is your audience.

The bigger the audience, the more impact one can make, to other people lives. Again you can use tools like Google Analytics or other monitoring products, but this time concentrate on the outcomes of particular actions.

Number of new users, increased amount of followers, number of retweets you name it. These metrics have other benefits that you can possibly leverage in the future. For instance granting you with the possibility to build products that will support your community.

Summary

There are two ways to use data from above points in daily life. First one is to monitor couple of different applications, which is technically easy, but sort of cumbersome in longer perspective. Second one, majority of applications, webpages, have an option to export data (for instance to Excel or .CSV format) or are giving developers access to API (application programming interface). Raw data from applications can be later used as source for “your” analytical application, be it Microsoft Excel, Qlik Sense, Power BI, Google Data Studio.

The benefit of second approach is that with a bit of work, you can upload all of your data from various source into one place. Of course, one can argue, that there is still manual work to be done (export, import etc.) but this is a good point to start, and maybe, just maybe a cool idea for a side-project.

What is your opinion? Do you have any other ideas as to how your own data can be used? Please let us know in the comments section down below.

Thanks,

Krzysiek

Leave a Comment

Twój adres e-mail nie zostanie opublikowany. Wymagane pola są oznaczone *