Chapter 2. A primer on web development and design

Interfaces are not black boxes with sovereign power inside. As an eventful, programmable and submersive milieu, the interface is a vortex of agencies, which maintains multiple facings, contrasting interior and exterior states, and relational pressures that sustain its threshold condition.1

2.1. Introduction

In August 1994, the sale of Sting’s Ten Summoner’s Tales CD via NetMarket marked the first ever online transaction.2 With the launch of the eBay3 and Amazon4 online marketplaces in 1995, e-commerce was born. In 2024, e-commerce is a global industry, projected to reach US$4.45tn in revenue.5 Aside from e-commerce websites and online marketplaces, businesses may sell to consumers via other digital channels such as mobile apps, social media platforms and shoppable links,6 which may be accessed through a host of devices, like computers and mobile devices (smartphones and tablets), or smart speakers like Amazon’s Echo.7 To stay competitive, online traders may offer goods and services via several digital channels (this practice is called ‘multichannel retail’) or even target their users across different channels (so-called ‘omnichannel retail’).8

Despite the growing diversity of sales channels and the associated changes in business strategies, e-commerce websites remain the dominant digital sales channel in the EU,9 and therefore represent a prominent site for the occurrence of consumer harms, including harms arising from the use of dark patterns. The automated detection of these harms requires some familiarity with basic web development terminology. Accordingly, this chapter starts by providing, in section 2.2, the technical jargon needed to comprehend the detection methodologies discussed in Chapter 8.

The same technological advances that have enabled sales via different channels and devices coupled with the forces of competition have changed not only the way an online sales channel is developed but also the actors involved in the process, and have reshaped the competitive factors in the e-commerce market. The second part of this chapter (section 2.3) zooms in on these technological, organisational and economic developments in order to highlight how the design of consumer-facing website interfaces may be influenced by competitive pressures and the organisational and technical resources available to online traders. It starts by looking, in section 2.3.1, at the main market actors behind consumer-facing e-commerce interfaces – online marketplaces and online stores – and the technological and organisational resources they may rely on for the development and maintenance of their digital infrastructures. Section 2.3.2 zooms in on user experience optimisation, a key aspect of the e-commerce website development process and of success in the market. Section 2.3.3 reflects on what the findings of this chapter mean for the consumer-friendliness of e-commerce interfaces.

2.2. A primer on web development

This section provides a short introduction to how websites work. It starts by defining websites and explaining the role of core web technologies in their functioning. The second sub-section explains how browsers render web pages, and describes what kind of data they generate in this process; this data can be extracted and analysed (more on that in Chapter 8). Lastly, this section looks at dynamic websites, which may present challenges for automated data extraction from websites.

2.2.1. Websites: Definition and building blocks

A website is a collection of interlinked web pages that are grouped under the same domain name.10 Web pages are documents on the World Wide Web. The core technologies powering up web pages are HyperText Markup Language (HTML), Cascading Style Sheets (CSS) and JavaScript.11

A. HTML

HTML is the standard markup language used to structure a web page and the content on it.12 The building blocks of web pages are HTML elements.13 There are various categories of HTML elements – text, links, forms, tables, lists and a variety of media, to name a few.14 HTML elements are enclosed in opening and closing tags, which denote the element name and state where the element begins or starts to take effect.15 Elements can be modified to behave in a certain way using attributes;16 for example, a checked attribute can mark an <input> checkbox element as checked by default. Attributes can also be used to provide metadata;17 for example, the only function of the class attribute is to assign an element to a group of elements to be targeted by CSS or JavaScript.18 Most attributes take the form of name–value pairs.19 Fig. 1 depicts the structure of an HTML paragraph (‘<p>’) element.

HTML elements can be nested, i.e. placed inside other elements to structure a web page. The entire document is wrapped in an ‘<html>’ element. The ‘<head>’ element of an HTML document contains elements denoting metadata: e.g. the page title, keywords that may be used by search engines, links to CSS or JavaScript files that need to be applied to the HTML, and any other data that is not considered web page content.20 Web browsers do not display the head’s content when the page is loaded; they display the contents of the ‘<body>’ element.21 Fig. 2 illustrates the basic structure of an HTML document.

B. CSS

CSS is a language for defining the presentation and appearance of an HTML document. CSS is used to alter the font, colour, size, layout and spacing of the web page content, or add decorative features, such as animations.22 The use of CSS facilitates the separation of content (HTML) and presentation – CSS style sheets can be applied to various pages on a website without altering content, and HTML content can be updated without interfering with its presentation, and can also be accessed on devices with different rendering methods (e.g. screen readers).23

CSS relies on rules. CSS rules are defined by specifying the style(s) to be applied to HTML elements or groups thereof. The element a rule applies to is termed a ‘selector’. The rule proper is called a ‘declaration’, and it consists of property–value pairs.24 Fig. 3 below illustrates the structure of a CSS rule.

The most common way to apply CSS to HTML documents is by referencing a separate file containing CSS via <link> elements; it can also be applied via an internal stylesheet in the document head or through in-line ‘<style>’ elements.25 Fig. 4 below shows how style can be added to the code in Fig. 2 in-line.

C. JavaScript

JavaScript is a programming language which is used to add interactivity to a web page. This presupposes dynamically modifying HTML and CSS;26 the specifics of how browsers achieve this are described in the next sub-section. Prior to the 1995 release of JavaScript, most web pages were static, i.e. they showed the same content all the time.27

JavaScript code is typically embedded or referred to (for example, when it is contained in an external file) in HTML documents via <script> elements.28 Fig. 5 shows how the text element from the previous examples is modified by JavaScript to change its colour to blue when the mouse is hovered over it.

2.2.2. How web pages are rendered

When a user navigates to a web page using a browser (the client) on a connected device such as a computer or a smartphone, the browser sends an HTTP request to the server storing the website resources. If the server approves the client’s request, it will start sending the files corresponding to the website, which the browser assembles into a web page in a process known as the ‘critical rendering path’.29 The browser will parse HTML first; in the meantime, the browser sends additional requests to the server for any external CSS and JavaScript files found in the <link> and <script> elements. The browser constructs an in-memory Document Object Model (DOM) tree out of the parsed HTML. This presupposes transforming each HTML element, attribute and piece of text into a DOM node in the DOM tree. The browser then processes the CSS rules and builds a CSS Object Model (CSSOM) tree, which is a DOM-like structure for CSS containing information on how to style the DOM. JavaScript can access and manipulate both the DOM and CSSOM trees to add dynamicity to a web page. The DOM and CSSOM trees are then combined into a render tree, which contains both the content and style information of all the visible content, and is used to compute the layout (dimensions and locations) of visible page elements. Lastly, once the layout phase is complete, the browser starts painting a visual representation of the page that the user can see and interact with on the screen. This visual representation is called a user interface (UI). If a user interaction with the UI leads to changes in the web page content, the affected element(s) will be repainted or the page will be re-rendered.30

2.2.3. Dynamic websites

In the early days of the internet, most websites were static, i.e. they were made up of hard-coded HTML files stored on the server, which would be delivered to the client as-is. This means that each user would see the same content, and the content would only change if the source files were modified manually. For any website that does not involve a handful of read-only, informative pages (e.g. a resume website), displaying all website content using different static pages is terribly inefficient.31 Accordingly, today most websites are dynamic, i.e. they use static templates built using HTML, CSS and JavaScript, and dynamically populate these templates with data stored on or generated by the server when needed, e.g. in response to user actions like scrolling or logging in, user preferences, and characteristics such as location.32

Upon initial loading of the page (the so-called ‘first meaningful paint’),33 a dynamic web page may not contain content. This is because the HTML file that is used in the first meaningful paint is actually a template, not the full web page. For example, a logged-in user accessing X.com’s ‘For you’ timeline will first see a loading icon (see Fig. 6), while relevant tweets (based on the accounts a user follows, content those accounts interact with and topics of interest)34 are being fetched from the server.

2.3. The market actors and forces shaping e-commerce websites

From a consumer law perspective, the e-commerce market actors that are relevant are those that consumers transact with: online marketplaces and online stores. However, if we look at the digital infrastructure enabling these transactions, a much more complex picture emerges: there is a web of third parties who provide the key components underlying consumer-facing user interfaces. While this section (and this book overall) focuses on online traders, it also provides a brief look into how these actors develop and maintain their digital infrastructure, which ultimately plays a role in whether they have the capacity to intentionally cause consumer detriment and, by extension, to breach regulations. This analysis is meant to be neither exhaustive nor empirical, but rather illustrative of the resource disparity amongst market players. The second sub-section zooms in on one of the crucial aspects of the e-commerce website development process – user experience optimisation, which is key to e-commerce success, yet may result, wittingly or not, in the emergence of potentially harmful design elements (dark patterns) in consumer-facing e-commerce interfaces.

2.3.1. Market actors

A. Online marketplaces: The case of Amazon

B2C online marketplaces are online platforms that intermediate transactions between third-party sellers and consumers. The intermediation of transactions may involve merely providing the technical infrastructure for their conclusion,35 or also order fulfilment services.36 Some online marketplaces (e.g. Amazon, Zalando, bol.com) not only intermediate transactions but also sell their own products directly to consumers. As the purpose of the discussion here is to illustrate the disparate resources of e-commerce market players, in what follows I will focus on the example of Amazon. A focus on Amazon also allows us to look at how the retail giant has shaped the competitive factors in the e-commerce industry.

There is more to online marketplaces than consumer sales or the intermediation thereof. Online marketplaces are also tech companies. As Werner Vogels, Amazon’s Chief Technology Officer, put it in 2016: ‘We may be a retailer, but we are a tech company at heart. When Jeff started Amazon, he didn’t start it to open [a] book shop’.37 While it’s hard to single out a single innovation that is responsible for Amazon’s success, Amazon’s dominance in the e-commerce market can definitely be at least in part credited to its long-term and continued commitment to innovation in e-commerce technology. In the early days of the internet, online shopping used to be a task fraught with uncertainty – fake online stores abounded.38 The introduction of the 1-click checkout technique, which allows customers to pre-save their payment details and complete a purchase through a single click on a ‘Buy now’ button, convinced early venturers into online shopping that they could trust Amazon with their payment data in exchange for less friction.39 According to some estimates, 1-click checkout and the ease of repeat purchases that it entails have earned Amazon billions in the process.40 Access to customer data at scale has enabled Amazon to devise and perfect other e-commerce innovations, such as its product recommender system. Amazon was a pioneer of item-based collaborative filtering in recommender systems,41 a method that has since been applied in other industries, such as video streaming, travel, news and advertising,42 and that is still widely used today – the 2003 paper on the method published by Amazon’s engineers was recognised by the IEEE Internet Computing journal in 2017 as the article that has best withstood the test of time.43

Amazon’s efforts to innovate are supported by the work of some 35,000 software engineers worldwide, according to some insiders.44 The retail giant’s commitment to innovation has arguably paid off – Amazon is the largest online retailer in the world,45 capturing 29% of e-commerce sales in the EU in 2022,46 and its European websites get over a billion monthly visits;47 in some EU MS, over 50% of internet users have an Amazon Prime membership.48 Its continuous innovation in the sphere of e-commerce technology, coupled with its innovations in logistics, which enable it to deliver products fast and conveniently,49 have also had lasting consequences for the e-commerce market in that they have transformed consumer expectations. Consumers expect an almost-instant and frictionless shopping experience, regardless of the industry or product.50 This transformation is often referred to as the ‘Amazon effect’ in marketeer circles;51 sub-section 2.3.2 explores what online stores can do to stay afloat in a market dominated by Amazon. Amazon’s technological breakthroughs also allowed it to reinvent its business model52 – once it had perfected a particular technology, such as its recommender system, it could start selling it as a service to other companies through Amazon Web Services (AWS).53 While the launch of AWS in 2006 was seen as a distraction by some industry analysts,54 nowadays most online businesses, including online stores, rely on third parties for some, if not all, elements of their digital infrastructure, as the next sub-section shows.

B. Online stores

Online stores are businesses that sell their own products directly to consumers. The non-Amazons of this world are, for the most part (over 99% in the EU according to some estimates), small and medium-sized enterprises (SMEs).55 For such companies, it is common to sell products to consumers via their own website or app, in some cases alongside offering them on an online marketplace like Amazon.56 While building websites was the prerogative of web developers in the early days of the internet, nowadays websites no longer have to be built from scratch (although they can be), and setting up a website without a web developer altogether is possible using low-code and no-code solutions like e-commerce website-building platforms. There are also a myriad of possibilities relying to various degrees on third-party pre-made solutions along this spectrum, and it is rare for any kind of website to be entirely self-contained nowadays: according to the HTTP Archive, a project that tracks how web page content is built and served, around 95% of websites use some kind of functionality provided by a third party.57 While the use of third-party services entails giving up control over some aspects of the website (as website developers may not directly control third-party code),58 and may have implications for user privacy,59 as well as website security60 and performance,61 it also reduces development costs (sometimes all that is required to implement the resource is copying and pasting HTML or JavaScript snippets from the third party’s website)62 and the website maintenance load for the site owner.63 As the HTTP Archive puts it, the availability of third-party services ‘enables even the most non-technical to be able to create and publish content to the web’.64 Notably, website design, development and maintenance costs can be significant, especially for a small online store: the design and development of an e-commerce website featuring up to 1,000 products can cost anywhere between $5,000 and $55,000, whereas the annual maintenance costs can reach $30,000.65 Accordingly, to cut costs, website developers will resort to third parties’ services to implement functionality for consent management, advertising, analytics (measuring visitors’ behaviour to evaluate marketing or website performance), marketing, online payment processing, and improving website performance and design, amongst other purposes.66

For retailers that are less resourced, various Software-as-a-Service (SaaS) e-commerce platforms have emerged in the last decade. According to the technology profiling tools Wappalyzer67 and builtwith,68 Shopify69 is the most popular e-commerce platform. E-commerce platforms like Shopify allow retailers to quickly set up shop on the web without writing a single line of code, using only a drag-and-drop builder. While this ease of use comes at the expense of customisation options, additional functionality can typically be added to a website using either first-party or third-party apps, potentially for a fee.70

2.3.2. User experience optimisation as a cornerstone of e-commerce success

While setting up an e-commerce website today is easier than ever, staying afloat in the e-commerce sector is more of a challenge: according to some reports, 90% of e-commerce start-ups fail within four months of launching.71 While small(er) retailers cannot compete with Amazon in broader customer experience terms – it is hard to beat the giant’s fulfilment speed – they may be able to do so in terms of user experience (UX).72

UX refers to the user’s experience with a digital product like an e-commerce website. There are several key aspects to a good UX: aesthetics, usability, website performance and responsiveness.73

Aesthetics refers to the visual appeal of a website’s UI. On average, users form a decision on the aesthetics of a website within 50 milliseconds of visiting it,74 and 38% of them will leave the website if they find its design unattractive.75

Usability deals with how easy an average user finds it to use the website in order to complete specific actions, such as purchasing a product.76 Simply put, if a user cannot find the product on the website, they cannot buy it.77 A website’s UI therefore has to function well alongside being beautiful (although studies have found that users tend to be more forgiving of minor usability shortcomings when a website is visually pleasing).78

Website performance is concerned with how quickly site content loads and renders in a web browser, and with how well it responds to user interaction.79 Bad performance may lead to website abandonment, or even bouncing (abandonment after visiting a single page).80 Users typically expect a website to load in two seconds or less, and will start abandoning slow websites at 3 seconds.81 Seventy-nine per cent of users would hesitate to return to purchase a product on a website they found slow.82

Responsive web pages are those that respond well to the needs and capabilities of the device they are accessed from.83 As the chapter introduction notes, nowadays users may access a digital sales channel like a website via multiple devices, with varying screen sizes, processing speeds and modes of interaction (e.g. touchscreens on smartphones). Responsive websites look good, work well and load quickly, irrespective of the device a user is using.84 Website responsiveness is becoming ever-more important as mobile devices like smartphones and tablets are becoming more and more popular amongst online shoppers.85

A negative user experience may thus affect an online trader’s performance, and retailers therefore have an incentive to optimise the user experience to maximise performance.86 Amongst the many performance indicators that may be used to gauge the success of an e-commerce website,87 conversion rate is typically described as one of the most (if not the most) important e-commerce success metrics.88 Conversion rate is the proportion of website visitors who make a purchase on the website.89 Conversion rate optimisation often involves the use of A/B tests.90 A/B tests are online controlled experiments in which users are split into groups that are shown different versions of a digital product – typically tweaks in the UI features of a website – in the search for the optimal design, i.e. the one that results in the highest conversion rate.91

Large tech companies like Amazon have their own internal A/B testing platforms,92 randomising hundreds of thousands of users to hundreds of experiments that run in parallel.93 Not all of these experiments are successful: about 50% of tested features fail to improve the metrics they were designed to improve, but every now and then they may yield a significant breakthrough, which is why big players are willing to foot the bill of innovation.94

There are a myriad of third parties that have brought the possibility of innovation to the ordinary online store, offering website owners the ability to track and analyse the behaviour of website users to investigate so-called ‘drop-off points’95 in the customer journey that may need adjustment96 and to run A/B tests to develop optimal solutions for these bottlenecks.97 A recent study estimates that around 24% of e-commerce start-ups make use of A/B testing services.98 The available solutions differ in terms of both pricing and the technical expertise necessary to make use of them. Some popular vendors of A/B testing services, like Optimizely,99 AB Tasty100 and VWO,101 offer visual editors that facilitate no-code modifications to the UI, allowing non-technical users to set up experiments; however, these editors typically only allow minor modifications to the UI design, such as changing the colours of buttons or editing text. AB Tasty and VWO also offer libraries of ready-to-use widgets that can be deployed and tested on websites, but access to these is typically reserved for users on more costly, enterprise-level priced tiers.102 Therefore, it still appears to be necessary to have some in-house UX design and web development capacity in order to design and implement testing variants. Even so, there is no guarantee that an A/B test will lead to useful innovation. A retailer’s website may simply have too little traffic for its experiments to produce reliable results: generally, it is advisable to have at least thousands – and ideally tens of thousands – of users.103 Further, understanding the results of the experiments also requires some kind of in-house expertise in statistical analytics,104 especially when we consider that prominent A/B testing platforms have been found to misrepresent results in the past.105 Lastly, website owners have to maintain a fine balance between website performance and running tests to optimise conversions – A/B tests can slow down a website, which may cause users to bounce, lowering the retailers’ chances of converting them.106

Instead of developing their own conversion tactics, website owners can also turn to agencies that provide A/B testing and conversion-rate-optimisation services,107 or implement tried-and-tested solutions developed by third parties. Companies like Nudgify offer apps meant to ‘increase trust and sales’ that can be easily integrated across a variety of platforms,108 whereas website-builder platforms offer apps with ready-made solutions for conversion.109

2.3.3. Summary and takeaways

Selling CDs on NetMarket in 1994 required modifying Mosaic, one of the first web browsers, to incorporate Pretty Good Privacy, an encryption program.110 Setting up an online store today is arguably a much easier task. As this section shows, online stores have a wide array of third-party tools at their disposal for setting up and managing their digital presence. Using some of these tools, such as drag-and-drop website builders, requires little to no in-house web development and design expertise, and therefore offers a cost-effective solution to what was once a task that could only be performed by highly skilled web developers. This ease of use does sometimes come at the cost of having complete control over one’s digital infrastructure: not all third-party solutions will be equally customisable. Further, setting up an online shop is only part of the equation. To stay competitive, businesses have to constantly work towards optimising the experience they offer to the users of their digital channels. The technical toolbox has also evolved in this regard: there are a myriad of website analytics tools that can offer businesses insights into how users engage with their websites, and A/B testing platforms offer the possibility to act on these insights. These methods can cater to users’ expectations in relation to website usability, aesthetics, performance and responsiveness; in other words, they can help devise beneficial innovation. The use of these techniques and the linking of user experience design goals to commercial measures of success also means, however, that some businesses may find ways to profit at the user’s expense through the implementation of user-unfriendly interface design choices – dark patterns. In other words, as Brignull points out, ‘the same research methodologies can be used to help or harm users’.111 That being said, design choices that do not serve users’ interests do not have to be deliberately crafted as such: sometimes they are, as Brignull explains, ‘just profitable accidents’,112 or, in other words, some random design feature that boosts conversion rates. At other times, dark patterns may be the result of a speedy digital product roll-out timeline – solutions that have not been subject to testing on representative user populations113 – or simply bugs. Further, as discussed in the previous section, innovating in terms of user experience design still requires some in-house technical and statistics expertise. This expertise may not be within (financial) reach for all small players, and they may resort to third-party services and ready-made solutions that they have little to no control over. What is more, some third-party service providers deploy manipulative practices against their (business) customers. A recent study revealed that consent management platforms (CMPs), intermediaries that facilitate the implementation of cookie-notice designs that are compliant with data protection law, may steer website owners towards paid plans and offer non-compliant default design templates.114 Another study found not only that 60% of the default templates offered by CMPs contained dark patterns, but also that the generation of privacy-friendly designs is in some cases impossible; where it is possible, it is often inconvenient, and sometimes only accessible for premium-tier users.115

That being said, not all businesses will be equally powerless or clueless when it comes to the implementation of dark patterns. As section 2.3.1 explains, big tech companies that engage in online sales have made innovation the central tenet of their corporate culture; they also have the in-house resources and expertise necessary to practise what they preach by testing an astonishing number of product design variations in the search for the one that will get users to do what the business wants them to do. Douglas Bowman, Google’s former visual design lead, famously quit in 2009 amidst the company testing 41 different shades of blue to determine which one performs better.116 Some design variations may also be knowingly designed for user detriment. In June 2023, the US Federal Trade Commission filed a complaint against Amazon for its maze-like cancellation process for Prime subscriptions.117 The complaint alleges – based on internal documents made public in 2022 by Business Insider118 – that Amazon and its leadership knowingly kept users trapped in their Prime subscription contracts by resisting user experience changes that would have simplified the process.119 Internally, the cancellation process was called ‘Iliad Flow’, a reference to Homer’s epic poem about the decade-long Trojan War,120 which suggests that Amazon’s main goal was to deliberately make the cancellation flow as hard as possible.121

Drawing a distinction between big tech companies and small businesses that sell to consumers online is arguably an oversimplification, but it is useful to illustrate that the technological and organisational resources used by online traders to deal with their digital infrastructure vary enormously. This disparity in resources could very well mean that there is a difference in both the capacity to design and deploy or prevent practices that may harm consumers (dark patterns), as well as in willingness to do so. I discuss the implications of this for the way we design regulation in Chapter 4. For now, we turn to Chapter 3 for a closer look at dark patterns.