Data processing and consumption are essential elements of the contemporary business world. Therefore, there are mysterious pieces of software, commonly abbreviated as APIs, whose role is fundamental in this traffic of information.
APIs (Application Programming Interface) are mechanisms for integration and communication between machines, services or programs. Through them, systems expose functionalities and/or data that can be consumed by other systems or services. Their essence is to be interfaces between systems to facilitate compatibility and interoperability, regardless of their operation or the technologies they implement.
Currently, there are a wide variety of API protocols in the industry with the aim of standardising communications between systems in some way. In this article, we will begin to explore OData v4 (Open Data Protocol), an OASIS standard that builds on the principles of RESTful APIs and allows you to expose data sources and perform queries in a very simple way.
REST APIS and OData APIS
First of all and for the sake of completeness, let’s briefly introduce what REST (Representational State Transfer) APIs are. These APIs define communication generally through the HTTP protocol using a set of standard operations such as GET, POST, PUT and DELETE to interact with resources. In their implementation they tend to be simple and scalable and use readable URLs to access data, taking advantage of HTTP methods and statuses (such as the famous ‘404 Not Found’) to perform and manage requests and responses. APIs that rigorously follow the REST protocol are commonly referred to as RESTful APIs.
On the other hand, the OData protocol was originally developed by Microsoft in 2007 and has become a widely adopted open standard. It aims to further facilitate interoperability between different applications by providing a standard for interaction with data sources. It achieves this primarily by building on a metadata standard associated with each source in CSDL.XML (Conceptual Schema Definition Language) format.
Thanks to them, the client can discover the structure of the exposed data without relying on any documentation or contact with the service maintainer. On top of this, OData defines a wide range of operations, including filtering, sorting and pagination that are transformed, for example, into SQL statements to be delivered to the data source. In addition, it also defines what the responses delivered by the services should look like.
Finally, although OData is often associated with relational databases, it can also be used with non-relational databases, thus covering a large number of data sources in our systems.
Metadata in OData v4
In this first article on the OData protocol, we are going to focus exclusively on explaining what is in the metadata of an OData v4 service, and we will also see how to access the data using some basic instructions. The CSDL metadata file will be divided into three parts as shown in the following image.
In addition, to better understand its details, we will break down the metadata with a case study, as the OData protocol is indeed very extensive and we are not going to cover everything. To do this, we will use the reference services on the OData v4 standard’s own website, where you can also download a Postman collection with some examples of how to interact with the API.
Types, properties and entities
Let’s imagine for a moment an external service trying to synchronise with this test OData API, the first thing it will probably do is ask for the metadata so it knows what to expect and how to interact with it. To do this, it will use the ‘metadata’ parameter on the root of the service, and the service will respond with the CSDL.XML file. This file, as mentioned above, is the service’s operations manual, which tells us how the data is structured and how we can request it. Let’s see now a first fragment of it after launching the following instruction.
GET basePath/TripPinRESTierService/$metadata<EntityType Name="Person">
<Key>
<PropertyRef Name="UserName" />
</Key>
<Property Name="UserName" Type="Edm.String" Nullable="false" />
<Property Name="FirstName" Type="Edm.String" Nullable="false" />
<Property Name="LastName" Type="Edm.String" MaxLength="26" />
<Property Name="MiddleName" Type="Edm.String" />
<Property Name="Gender" Type="Trippin.PersonGender" Nullable="false" />
<Property Name="Age" Type="Edm.Int64" />
<Property Name="Emails" Type="Collection(Edm.String)" />
<Property Name="AddressInfo" Type="Collection(Trippin.Location)" />
<Property Name="HomeAddress" Type="Trippin.Location" />
<Property Name="FavoriteFeature" Type="Trippin.Feature" Nullable="false" />
<Property Name="Features" Type="Collection(Trippin.Feature)" Nullable="false" />
<NavigationProperty Name="Friends" Type="Collection(Trippin.Person)" />
<NavigationProperty Name="BestFriend" Type="Trippin.Person" />
<NavigationProperty Name="Trips" Type="Collection(Trippin.Trip)" />
</EntityType>
<EntityType Name="Airline">
<Key>
<PropertyRef Name="AirlineCode" />
</Key>
<Property Name="AirlineCode" Type="Edm.String" Nullable="false" />
<Property Name="Name" Type="Edm.String" />
</EntityType>
<EntityType Name="Airport">
<Key>
<PropertyRef Name="IcaoCode" />
</Key>
<Property Name="Name" Type="Edm.String" />
<Property Name="IcaoCode" Type="Edm.String" Nullable="false" />
<Property Name="IataCode" Type="Edm.String" />
<Property Name="Location" Type="Trippin.AirportLocation" />
</EntityType>As we can see in this snippet of the CSDL.XML file, below the first four header lines where the OData version and the Namespace name are defined (we have excluded them), there are three different EntityTypes: ‘Person’, ‘Airline’ and ‘Airport’. As the name indicates, each of them defines the structure of an entity type present in the data source.
Each EntityType has defined a Key, a Porperty and, in the case of the entity ‘Person’, also a NavigationProperty. The keys always refer to the name of one of the entity’s properties and may contain several of them. This is because the properties are actually the columns or fields of the entity in question that include, in addition to the name, the data types and other optional information such as its maximum length (in the case of text) or whether it can contain null values.
On the other hand, with the NavigationProperty we define links to other entities, as in the case of ‘BestFriend’ being a link to another instance of type ‘Person’, or ‘Friends’ being a link to a collection of entities of type ‘Person’. To obtain the data from the ‘Person’ table, we are going to add the word ‘People’ to the root (the reason for this name change will be explained later):
GET basePath/TripPinRESTierService/People{
"@odata.context": "https://services.odata.org/TripPinRESTierService/(S(3mslpb2bc0k5ufk24olpghzx))/$metadata#People",
"value": [
{
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
{"Person2"},
{"Person3"},
"...",
{"PersonN"}
]
}The API has responded with a message with two main elements: the OData context and the value of the response. The context is nothing more than the location of the data schema in the metadata. The response value is the data itself, which in this case contains all the ‘Person’ entries in the data source. Using the entity key in parentheses we can request individual ‘Person’ entities, for example using People(‘russellwhyte’) to get just the ‘Person’ with UserName == russellwhyte.
So, as we can see, the values for all the entity’s Property, which are its fields, do not appear at all with respect to the NavigationProperty. This is because by default these are not shown, as doing so could bring a lot of problems. To show them we have to use one of the query parameters, the ‘expand’, being able to ask for more than one of them separating them by commas. For example:
GET basePath/TripPinRESTierService/People('russellwhyte')?$expand=BestFriend,Friends{
"@odata.context": "https://services.odata.org/TripPinRESTierService/(S(e2canygeoavavzu0nrasd4ov))/$metadata#People(BestFriend(),Friends())/$entity",
"UserName": "scottketchum",
"FirstName": "Scott",
"LastName": "Ketchum",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Scott@example.com"
],
"FavoriteFeature": "Feature1",
"Features": [],
"AddressInfo": [
{
"Address": "2817 Milton Dr.",
"City": {
"Name": "Albuquerque",
"CountryRegion": "United States",
"Region": "NM"
}
}
],
"HomeAddress": null,
"BestFriend": {
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
"Friends": [
{
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
{
"UserName": "ronaldmundy",
"FirstName": "Ronald",
"LastName": "Mundy",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Ronald@example.com",
"Ronald@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
}
]
}Here we notice two things. Firstly, we see that the message no longer has the ‘value’ field, but directly adds the ‘@odata.context’ field to the entity properties. This is because now we are no longer asking for a collection of entities, but a single element with the parentheses. Secondly, we can see that this time the fields ‘BestFriend’ and ‘Friends’ do appear in the response, being entities or collections of type ‘Person’, but without their NavigationProperties.
This example gives us a very clear clue of the danger of always showing the NavigationProperties. I am a friend of my friend who is my friend and I am his friend … We would enter an infinite loop of mutual references.
To close this block, let’s now talk about the data types available for Property and NavigationProperty. The standard defines the most common in the programming world as String, and they are named using the acronym EDM (Entity Data Model) before the name (Edm.String). However, it is very important to always check the type compatibility between our data sources and OData types. In addition to the standard ones, OData allows you to define your own types, including EntityTypes, as we have already seen with the NavegationProperty, EnumTypes and ComplexTypes. In the next part of the file we see some examples.
<EnumType Name="PersonGender">
<Member Name="Male" Value="0" />
<Member Name="Female" Value="1" />
<Member Name="Unknown" Value="2" />
</EnumType>
<ComplexType Name="Location">
<Property Name="Address" Type="Edm.String" />
<Property Name="City" Type="Trippin.City" />
</ComplexType>
<ComplexType Name="City">
<Property Name="Name" Type="Edm.String" />
<Property Name="CountryRegion" Type="Edm.String" />
<Property Name="Region" Type="Edm.String" />
</ComplexType>
<EntityType Name="Employee" BaseType="Trippin.Person">
<Property Name="Cost" Type="Edm.Int64" Nullable="false" />
<NavigationProperty Name="Peers" Type="Collection(Trippin.Person)" />
</EntityType>
EnumTypes, as you can see, are used to define enumerations. They are very useful for fields with unique and limited values. On the other hand, ComplexTypes are very similar to EntityTypes with the difference that they do not have any Key and that they cannot be referenced by themselves, they always have to be a type of a field of an EntityType. In the example above, we can see how even a ComplexType like ‘City’ can be used inside another ComplexType like ‘Location’. We can also see how new EntityTypes can be defined using other EntityTypes as a base, for example the case of ‘Employee’.
Actions and Functions
The OData standard relies on CRUD (Create, Read, Update, Delete) operations to interact with data sources, but it also allows the capabilities of data sources to be extended through the use of Actions and Functions, defined and integrated into the data source.
The main difference between actions and functions is that actions can modify the state of the data source, such as updating a record, while functions cannot. Also, given a set of parameters and a database state, functions will always return the same results, whereas actions do not have to.
Finally, functions are invoked using the HTTP GET method, while actions are invoked using the POST method. This is because, in general, functions are used to read data, while actions can insert, update or delete data. Let’s look at some examples of both in the metadata:
<Function Name="GetPersonWithMostFriends">
<ReturnType Type="Trippin.Person" />
</Function>
<Function Name="GetFriendsTrips" IsBound="true">
<Parameter Name="person" Type="Trippin.Person" />
<Parameter Name="userName" Type="Edm.String" Nullable="false" Unicode="false" />
<ReturnType Type="Collection(Trippin.Trip)" />
</Function>
<Function Name="GetInvolvedPeople" IsBound="true">
<Parameter Name="trip" Type="Trippin.Trip" />
<ReturnType Type="Collection(Trippin.Person)" />
</Function>
<Action Name="ResetDataSource" />
<Action Name="UpdateLastName" IsBound="true">
<Parameter Name="person" Type="Trippin.Person" />
<Parameter Name="lastName" Type="Edm.String" Nullable="false" Unicode="false" />
<ReturnType Type="Edm.Boolean" Nullable="false" />
</Action>
<Action Name="ShareTrip" IsBound="true">
<Parameter Name="personInstance" Type="Trippin.Person" />
<Parameter Name="userName" Type="Edm.String" Nullable="false" Unicode="false" />
<Parameter Name="tripId" Type="Edm.Int32" Nullable="false" />
</Action>In this extract from the metadata file we see three functions and three actions. In the two types of methods we have defined the name and type of the parameters, as well as the types for the data they return. All this if indeed these methods need parameters or answer something in the form of a message.
For example, the action ‘ResetDataSource’ presumably returns the database to a base state, without returning any message or requiring any parameter. Both actions and functions can have the boolean tag ‘IsBound’ defined, which by default will be false. This tag indicates whether the function or action is bound to a particular entity type, which the standard always associates with the type of the first parameter of the function.
In addition, to avoid ambiguities when invoking a function or action, there is also the ‘EntitySetPath’ tag, which points to the entity on which we can invoke the methods. To call some of these functions and actions we would use URLs of this type:
GET basePath/TripPinRESTierService/GetPersonWithMostFriends
GET basePath/TripPinRESTierService/People('russellwhyte')/GetFriendsTrips(userName='scottketchum')
POST basePath/Trippin.svc/People(UserName='russellwhyte')/UpdateLastName
Content-Type: application/json
{
"lastName": "Smith"
}Entity container
As a final point in our journey through OData metadata, let’s talk about the entity container. This is where the magic really happens and all the pieces of the puzzle fit together.
Containers have a name, in this case ‘Container’, and within them are organised and defined the resources that are actually accessible from the outside with the API. So anything that is not in there, however defined in the CSDL file, does not exist from the point of view of the service consumer. Let’s look at the EntityContainer of this API:
<EntityContainer Name="Container">
<EntitySet Name="People" EntityType="Trippin.Person">
<NavigationPropertyBinding Path="BestFriend" Target="People" />
<NavigationPropertyBinding Path="Friends" Target="People" />
</EntitySet>
<EntitySet Name="Airlines" EntityType="Trippin.Airline">
<Annotation Term="Org.OData.Core.V1.OptimisticConcurrency">
<Collection>
<PropertyPath>Name</PropertyPath>
</Collection>
</Annotation>
</EntitySet>
<EntitySet Name="Airports" EntityType="Trippin.Airport" />
<Singleton Name="Me" Type="Trippin.Person">
<NavigationPropertyBinding Path="BestFriend" Target="People" />
<NavigationPropertyBinding Path="Friends" Target="People" />
</Singleton>
<FunctionImport Name="GetPersonWithMostFriends" Function="Trippin.GetPersonWithMostFriends" EntitySet="People" />
<FunctionImport Name="GetNearestAirport" Function="Trippin.GetNearestAirport" EntitySet="Airports" />
<ActionImport Name="ResetDataSource" Action="Trippin.ResetDataSource" />
</EntityContainer>EntitySets are nothing more than the collections of entities of a particular type available to the API. EntitySets are analogous to tables in a relational database. As we see in this example, the first EntitySet is ‘People’, just as we accessed entities of type ‘Person’ at the beginning of this article. Endpoints always refer to EntitySets, not EntityTypes. We can see that ‘People’ is a collection of ‘Person’ and, in addition, we can see how it is necessary to define also the names and types of the NavigationProperties.
Singletons, as the name suggests, are individual instances of entities within the server. They are defined separately because they are not collections of entities.
Finally, and in the same way as everything else, for a function or action to be available from the outside, it must be included in the entity container with a reference to its name and the EntitySet it uses.
Conclusion and next steps
In this article we have taken a first look at the OData v4 standard, introducing the CSDL metadata file in OData v4 and what information we can get from it. We talked about the different entities, types and properties, the functions and actions, and finally, we discussed how all these pieces fit together in the entity container to expose it to the outside world.
In the next installment dedicated to OData v4, we will talk about query syntax, query parameters, and other details that have been left out of this article.
Until next time, and happy data management!
