We know what’s beautiful: openness, abundance, clean systems, sharing, trust. Yet we work every day doing the opposite: scraping, hiding, competing, building complexity, surveilling.
some big part of what i did in data-related task in corporate wagie is competitor analysis. sometimes i really hate it. its a futile effort, because i often make a hypothesis, what if the dimension of money doesn’t not exist, but the things is money always matter.
when my friends go to big name company only do something like competitor analysis, is just crazy. you spend most of the time learning math, lots of theory that moves the world, but you ended up just doing a bunch of data transformation and then present it to high level for decision making.
but its not our fault, indonesia is social country, most (or i) just live to make impression that we live a good life, make a perfect cv, personal branding, every effort is intended to make ourselves not look miserable.
competitor analysis department exist because of fear. my fallacy: main business objective is only increase profit and decrease cost, other than that is just a noise. because organization is fear that their market get eaten, if consumer change their preference, how company can survive? how top level management become richer?
and to do competitor analysis, you have to get data. in digital world, you compare everything your competitor do, what they have and what we should adapt, everyone try to be the number one, try to monopolize, try to become super app. lots of bots mimic human, and human mimic robot.
“I don’t like all these bots trigger our analytics and looking like users on our site! And it adds more load to our servers! Let’s pay cloudflare 15k/mo to block the bots”
Now all the web scrapers switch from a lightweight request, get the html/api, move on… to now using a full browser to bypass cloudflare. Adding MORE load by loading all unnecessary APIs, ALL of the images and videos, and looking even more like real users.
Congratulations company. You paid to get an even worse result. CTOs can be absolute morons.
we often try to protect our data, try to steal their data, in order to know, in order to survive. its basically fear. and fear is fundamental function that move human to do that. and its because we cant align what human value really is.
the ugly part is the financial system, we dont know any other system than that. just because we dont know the alternative, we just try to continue. because it invent enjoyment distractions from consumption process, that makes us continue this idiocracy, to continue supporting the government, while everytime we’re robbed. we dont really know what makes them get more in their industry than what we get. like if you compare the leadership individual, with laborer in construction or market, what do you think the leadership do until they get multiple of what the laborer do? and we never ask or protest about this, because we live in the middle of comfort, in the middle of cheap happiness from consumption, either material or mental or spiritual. and someone in the lower level just dont have enough energy to think and reflect.
but i think someone has do the dirty work, scraping, cleaning the data. to do things. thats why i hate when theres lot of unstructured data. the process that should be easy and save lots of time, they made it become so hard. why? because they really afraid of not being useful, that if they do something efficient and make easy for others, they will just be replaced, and thrown away.
thats why if you dev and do clean code, that make everything easy, that make the process of development faster and better, when its done and just need a little maintenance, you will be erased, no matter how good you are. thats why government really like the complexity of bureaucracy because by that they will keep their job.
and often you see slower thing is not a bug, but it is a feature. its a feature created by them because we are human is so afraid, we only want to survive. thats why lots of company hide their data, dont publish their data, make it hard for people to get. they afraid of being replace, they afraid not survive. even in the act of afraid, they somehow know that there is some benefit that it will prevent them from DDOS lets say, for protection.
what are the reasons from most afraid people? they will said, “ohh i do this for protection” protection is only someone who is really afraid. someone who assume that there are always bad actor. they know themselves that they can be the bad actor, its a mirror. we are as human mostly doing this, until there’s someone who shows what abundant is. when creation is easy, not only in the world of bits, but in the biological and chemistry level.
how long we can wait till that happen? or its just a only an imagination?