What exactly are Web APIs? Why are Web APIs so popular and widely used? Let’s first explore what APIs are so you can understand how they evolved into Web APIs.
API stands for Application Programming Interface, and it lets developers integrate any two parts of an application or any different applications together. It consists of various elements such as functions, protocols, and tools that allow developers to build applications. A common goal of all types of APIs is to accelerate the development of applications by providing a part of its functionality out-of-the-box, so developers do not have to implement it themselves. There are APIs for all types of systems, including operating systems, libraries, and the Web.
APIs that work at the operating system level help applications communicate with the underlying layers and with each other by following a set of protocols and specifications. An excellent example of such a specification is the POSIX, short for Portable Operating System Interface. By following the POSIX standards, applications that are compiled to work on a particular operating system also work on other systems that follow the same criteria. Software libraries also play a significant role in creating compatibility between different systems.
APIs define the way software libraries behave. Applications that interact with a library should follow the set of rules specified by the API. This approach makes it easy for software developers to write applications that communicate with different libraries without having to rethink their strategy, as long as all the libraries follow the same API. Another advantage of this method is how easy it is to consume the same library using different programming languages, as long as the consumer application can interact with the API.
To be able to communicate with applications on remote machines, developers use remote APIs. By using remote APIs, developers make it easy for applications to interact with remote resources and systems in a standardized fashion. A good example of such a standard is ODBC, or Open Database Connectivity, which allows applications to consume different databases by accessing the same functions. The database, in this case, appears to be local to the developer, while it is in fact remote. The ODBC standard deals with all low-level aspects of connectivity, error handling and communication. Another type of remote API that is related to code execution is the Java Remote Method Protocol. In this case, instead of making a remote database appear local, it makes remote functions and objects available as if they were local to the caller application. Developers can then manipulate remote objects as if they were available locally without having to deal with any communication details.
A Web API is a unique type of interface where the communication takes place using the Internet and Web-specific protocols. Much like remote APIs make remote resources appear as local, Web APIs do the same thing for resources available on the Web. In fact, Web APIs started to become popular with the advent of internet services that let users store content online. In general, you serve Web APIs through an HTTP interface. The API itself defines a set of endpoints, request messages and response structures. It is a standard approach also to identify the supported response media types. XML and JSON are two favorite examples of response media types that can be easily interpreted by API consumers. While initially Web APIs were also called Web services, nowadays the use of the latter form signals that the API is RESTful, as opposed to following the SOAP standard.
The origin of Web APIs can probably be tracked down to the root of UNIX itself and how different applications, or processes, used to communicate with each other. What we call UNIX is, in fact, a group of operating systems with the same root: the AT&T Unix. This initial version was created in the 1970s by Ken Thompson and Dennis Ritchie who were at the time working at the Bell Labs research center. After the first version had been released, AT&T chose to license UNIX to third parties. This type of license made UNIX, and its multitude of variants, one of the most used operating systems. Among the different flavors, the most notable for having contributed to the history of Web APIs is the Sun Microsystems Solaris.
| Ken Thompson (sitting) and Dennis Ritchie at a PDP-11 minicomputer, circa 1970 |
From the beginning, UNIX was characterized for being a modular system where different applications, or processes, are built simplistically with the goal of having them working together. This approach, also known as the Unix philosophy, is one of its biggest strengths and key to the development of Web APIs. The system available on UNIX for sending and receiving messages from different processes is called IPC, or Inter-process communication. IPC is, in fact, a group of APIs that let developers coordinate how different concurrent processes are executed. IPC make this coordination possible by offering various flavors of inter-process communication such as pipes, message queues, semaphores, shared memory, and sockets. All communication methods, except sockets, are limited to processes running on the same server.
Taking advantage of the way network sockets operate, Sun Microsystems developed a way to communicate with remote processes called Remote Procedure Calls or, in short, RPC. It was initially implemented in the 1980s as part of Sun’s Network File System project and follows the calling conventions used in UNIX and the C programming language. RPC became quite popular because it allowed any running application to send a request to another application that would reply back with the result of the requested operation. Both messages and responses are serialized using the External Data Representation format or, in short, XDR. This approach decouples message producers from consumers by using a standard format that is understood by both. The RPC protocol can deliver messages with XDR payloads using either UDP or TCP, letting it work on different types of networks.
It took about ten years to get from the initial implementations to official publication in the form of an RFC. The RPC protocol was published in 1995 as RFC 1831 and has evolved through different versions until 2009 when RFC 5531 was published to describe it. In that year, Sun changed the license used by the RPC protocol to the standard 3-clause BSD making it available to use freely by anyone. Nowadays, most flavors of the UNIX operating system offer some type of RPC natively. Microsoft Windows also supports RPC officially through their Services for UNIX product and third parties with a series of implementations for programming languages such as C, C++, Java, and .NET.
Even though RPC has found great popularity, mostly because of it being considered lean and easy to implement and use, it is not the best protocol to use in highly heterogeneous network environments. RPC issues have to do with parameter passing and marshaling in situations where clients and servers are not written in the same programming language. This is where other protocols that abstract messages and parameters have started to thrive.
Service-oriented architecture or, in short, SOA, was thus born and became the de-facto standard for building cooperation among applications running on heterogeneous environments. More or less at the same time, different Internet public services were gaining popularity. Among them, a service was particularly grabbing the attention of the broader audience outside of the academic world: the World Wide Web.
The World Wide Web, or WWW, was born in 1989 by the hands of Tim Berners-Lee, an English scientist, and became the primary way of accessing information and communicating online. During its initial years, the Web consisted simply of interconnected pages where you could consult information. Information was updated by hand by what were called at the time Webmasters, or the people responsible for maintaining the Web pages. Over time, and with the rise of commercial Web initiatives, some different services were created that would let you upload and serve personal information such as photos, and blogs, and other types of multimedia. These services created the need to build desktop applications that would let users interact with such services more efficiently. While initially those applications were meant to be used to download information, with time they also allowed users to upload content.
| Sir Tim Berners-Lee invented the World Wide Web in 1989, Image © CERN |
The communication between those content creation desktop applications and the Web services that were being launched was the precursor of what we now call Web APIs. One of the most popular early stage Web API was Flickr’s. Flickr was a popular photo-sharing service that would let developers interact with it through a Web API where they could upload, download, list and search photos from a single user or the whole service.
On the more closed side of the software industry, other protocols started to emerge, with the goal of simplifying the life of developers and integration designers. One such protocol that gained significant popularity probably because of its natural integration with existing Microsoft tools was the Simple Object Access Protocol, or, in short, SOAP. SOAP, in fact, became the number one way to integrate different Microsoft products without much effort.
More or less at the same time, another approach was taking shape. This time the goal was to reuse as much as possible the features of HTTP while adopting a view more in line with the needs of Web services. Hence, the Representational State Transfer architectural style, or REST, was born. Compared to SOAP, REST is much easier to understand and process, and a natural winner on the open Web because it does not need so many constraints to operate as SOAP does. Because of that, another protocol was created. This time the goal was to increase the control over what is transmitted so that the information could be validated reproducibly. Google’s Remote Procedure Call, or in short gRPC, was born and it started to be used by almost all of Google’s open Web APIs.
If you have read this far, you should now know what APIs are and how they have evolved into what we call Web APIs — actually, we simply drop the word “Web” and just use “APIs.”
You should also have gained knowledge about some of the existing approaches to serving and interacting with Web APIs. There are much more protocols than the ones covered in this article, and new ones are being released all the time.
How are you using APIs these days? Feel free to share your experience.
