Submission
Privacy Policy
Code of Ethics
Newsletter

Surveillance Capitalism—What Do We Really Use Artificial Intelligence For?

As the rise of artificial intelligence (AI) and big data-based technologies continues, their impact on our everyday lives will become more and more visible. At the same time, AI-based developments are not only affecting our lives in terms of convenience but are also deeply embedded in economic decision-making and social dynamics. Companies are adopting new strategies, of which personalized pricing seems to be one of the most impactful innovations. According to its proponents, this “technology” (or rather method) will make the customer experience more personal and convenient. But, of course, as usual, a series of ethical and economic problems are likely to follow.

The essence of “surveillance capitalism” is that big tech companies collect huge amounts of data about users’ online activities and use this data to shape and predict patterns of behavior. The concept was introduced by Harvard University professor Shoshana Zuboff. In its formulation, this new capitalist system is based on the “equal appropriation” of private life and human experience. Users’ activities are treated as data (often without their knowledge or consent) and then used for commercial purposes. This system is described as a “one-way mirror”, where companies see and influence everything, while users are unaware of what is happening with their data. A prime example is Google, which was one of the first to understand why it is worth collecting more data than is directly usable at any given moment, and how to market that data to others. Indirectly, this phenomenon led to the emergence of data brokers and a specific data industry.

These data have a “classic range”, which is often used as an example in similar lists. These include demographic information, purchase history, online browsing habits, social media activity, etc. However, as machine learning becomes more sophisticated, new elements are being added to this list, such as biometrics (facial recognition, fingerprints, voice recognition or even retina scanning). This makes it easy to match extra data to individuals in real time, such as that provided by emotion recognition. But it also includes the field of predictive analytics, which can even draw conclusions about the future behavior of individuals, based on the past actions of individuals who exhibit similar behavior. The list is not exhaustive, and it should be remembered that the more data we allow to be collected, the more the potential uses of that data will increase (often in unforeseen directions).

A good example of what all this data can be used for is personalized pricing, which is based on the use of individual customer data. The method essentially consists of selling the same products or services at different prices to different customers, depending on the price sensitivity and willingness to buy predicted by the algorithm.

The economics concept of reservation price refers to the maximum price that a consumer is willing to pay for a good or service, or the minimum price that a seller would accept for the sale of a good or service. In other words, the reservation price is the highest price that a buyer would still find worth buying, or the lowest price that a seller would still accept to buy a product. In the market, when setting prices, sellers seek to price above the consumers’ reservation price, while consumers seek to price below the sellers’ reservation price. Prices reach equilibrium when buyers and sellers are willing to transact at the market price.

The problem is that setting the reservation price is a very complex process for the seller. Sellers gather data from customer behavior, preferences, and demographics through various channels, such as online search history, social media activity, and buying habits. The data is used to make estimates of how much a customer is willing to pay for a product or service using AI and machine learning. In addition, a common method is to segment the market into different consumer groups, targeted by different price categories. This is based on demographics, income levels, geographic location, or other relevant characteristics. For higher-income or loyal customers, sellers often offer a higher price because they assume that these customers are willing to pay more.

Of course, all this means nothing if the data available on the person is incomplete, unreliable, or simply insufficient. At present, personalized pricing is not a common practice and is often based on very rough estimates. Airline ticketing, for example, is at the forefront of adapting this technique. Anyone who has ever traveled by air will have noticed that when you often search for air tickets for a particular route or flight, the system detects this and assumes that you are likely to want to buy that ticket. The algorithm can then infer that the customer’s reservation price is higher than average because of intense interest. The system then gradually raises the price, assuming that the customer is willing to pay more as he continues to search for the same flight. The algorithms in the background are essentially trying to calculate what price the customer still finds acceptable and place the purchase price just below that level. So they try to bring the price closer to the user’s reservation price.

The point of person-specific pricing in this context is that if the reservation price can be estimated efficiently for each person, it leads to higher profits than if only the value calculated for larger (e.g. demographic) groups is considered valid for all persons in those groups.

Personalized pricing currently operates mostly in the online space, where companies can efficiently collect data on customers. It is already widespread in e-commerce, in the travel industry mentioned above, or even in the insurance sector. Presumably, its take-up will continue to be limited to the online space, as in physical shops, where customers can interact instantly, this would lead to immediate conflicts. And no seller wants that (because of bad marketing). In this form, therefore, one of the requirements of the technology is that, in addition to the data collected, the seller must also be able to hide what price they are offering to others.

Moving away from the “price discrimination” just described, and back to surveillance capitalism, the biggest problem with the phenomenon is that it is based not only on data collection but also on the manipulation of user behavior. Through the analysis of data, these companies can influence people’s decisions, which raises serious ethical and democratic problems. Democracy and individual autonomy may be weakened if people are no longer able to make decisions completely freely because digital platforms predetermine their behavior. This “data-driven social governance” will lead to new inequalities and concentrations of power that could threaten social well-being and democratic institutions in the long term.

The development of AI continues to gain momentum, and the technology is expected to become increasingly embedded in economic and social processes. In the future, personalized pricing and other techniques based on data collection and profiling will become even more sophisticated as AI systems become able to analyze most aspects of human behavior and predict market trends with increasing accuracy. At the same time, sustainability and ethical issues of technological development will be key to ensuring that this technology truly benefits society.

An inescapable issue for the prospects will be the cooperation of companies and governments to ensure the responsible use of AI, with the former not necessarily having an interest in doing so. We will only be able to reap the benefits of AI if we can balance technological progress with the protection of consumer rights and environmental sustainability.


István ÜVEGES is a researcher in Computer Linguistics at MONTANA Knowledge Management Ltd. and a researcher at the HUN-REN Centre for Social Sciences, Political and Legal Text Mining and Artificial Intelligence Laboratory (poltextLAB). His main interests include practical applications of Automation, Artificial Intelligence (Machine Learning), Legal Language (legalese) studies and the Plain Language Movement.