Data processing and consumption are essential elements of the contemporary business world. Therefore, there are mysterious pieces of software, commonly abbreviated as APIs, whose role is fundamental in this traffic of information.
APIs (Application Programming Interface) are mechanisms for integration and communication between machines, services or programs. Thanks to them, systems expose functionalities and/or data that can be consumed by other systems or services. Their essence is to be interfaces between systems to facilitate compatibility and interoperability, regardless of their operation or the technologies they implement.
What is the OData protocol?
Currently, there are a wide variety of API protocols in the industry with the aim of standardising communications between systems in some way.
The OData v4 (Open Data Protocol) is an OASIS standard that builds on the principles of RESTful APIs. It also allows queries to be performed in a very simple way.
Introduction to REST APIs and OData APIs
First of all, for the sake of completeness, let’s briefly introduce what REST (Representational State Transfer) APIs are. These APIs define communication generally through the HTTP protocol using a set of standard operations such as GET, POST, PUT and DELETE to interact with resources. They tend to be simple and scalable and use readable URLs to access data, taking advantage of HTTP methods and statuses (such as the famous ‘404 Not Found’) to perform and manage requests and responses. Therefore, APIs that rigorously follow the REST protocol are commonly referred to as RESTful APIs.
On the other hand, the OData protocol was originally developed by Microsoft in 2007 and has become a widely adopted open standard. It aims to further facilitate interoperability between different applications by providing a standard for interaction with data sources. It achieves this primarily by building on a metadata standard associated with each source in CSDL.XML (Conceptual Schema Definition Language) format.
Thanks to them, the client can discover the structure of the exposed data without relying on any documentation or contact with the service maintainer. On top of this, OData defines a wide range of operations, including filtering, sorting and pagination that are transformed, for example, into SQL statements to be delivered to the data source. In addition, it also defines what the responses delivered by the services should look like.
Finally, although OData is often associated with relational databases, it can also be used with non-relational databases, thus covering a large number of data sources in our systems.
Metadata in OData v4
In this first article on the OData protocol, we are going to focus exclusively on explaining what is in the metadata of an OData v4 service. We will also see how to access the data using some basic instructions. As shown in the following image, we divide the CSDL metadata file into three parts.
On the other hand, to better understand its details, we will break down the metadata with a case study, as the OData protocol is indeed very extensive and we are not going to cover everything. To do this, we will use the reference services on the OData v4 standard’s own website. There, you can also download a Postman collection with some examples of how to interact with the API.
Types, properties and entities
Let’s imagine for a moment an external service trying to synchronise with this test OData API. Probably, the first thing it will do is ask for the metadata so it knows what to expect and how to interact with it. To do this, it will use the ‘metadata’ parameter on the root of the service, and the service will respond with the CSDL.XML file. As we already pointed out, this file is the service’s operations manual. It tells us how the data is structured and how we can request it. Let’s see now a first fragment of it after launching the following instruction.
GET basePath/TripPinRESTierService/$metadata<EntityType Name="Person">
<Key>
<PropertyRef Name="UserName" />
</Key>
<Property Name="UserName" Type="Edm.String" Nullable="false" />
<Property Name="FirstName" Type="Edm.String" Nullable="false" />
<Property Name="LastName" Type="Edm.String" MaxLength="26" />
<Property Name="MiddleName" Type="Edm.String" />
<Property Name="Gender" Type="Trippin.PersonGender" Nullable="false" />
<Property Name="Age" Type="Edm.Int64" />
<Property Name="Emails" Type="Collection(Edm.String)" />
<Property Name="AddressInfo" Type="Collection(Trippin.Location)" />
<Property Name="HomeAddress" Type="Trippin.Location" />
<Property Name="FavoriteFeature" Type="Trippin.Feature" Nullable="false" />
<Property Name="Features" Type="Collection(Trippin.Feature)" Nullable="false" />
<NavigationProperty Name="Friends" Type="Collection(Trippin.Person)" />
<NavigationProperty Name="BestFriend" Type="Trippin.Person" />
<NavigationProperty Name="Trips" Type="Collection(Trippin.Trip)" />
</EntityType>
<EntityType Name="Airline">
<Key>
<PropertyRef Name="AirlineCode" />
</Key>
<Property Name="AirlineCode" Type="Edm.String" Nullable="false" />
<Property Name="Name" Type="Edm.String" />
</EntityType>
<EntityType Name="Airport">
<Key>
<PropertyRef Name="IcaoCode" />
</Key>
<Property Name="Name" Type="Edm.String" />
<Property Name="IcaoCode" Type="Edm.String" Nullable="false" />
<Property Name="IataCode" Type="Edm.String" />
<Property Name="Location" Type="Trippin.AirportLocation" />
</EntityType>EntityTypes and Property in OData v4 metadata
As we can see in this snippet of the CSDL.XML file, below the first four header lines where the OData version and the Namespace name are defined (we have excluded them), there are three different EntityTypes: Person, Airline and Airport. As the name indicates, each of them defines the structure of an entity type present in the data source.
Each EntityType has defined a Key, a Porperty and, in the case of the entity Person, also a NavigationProperty. The keys always refer to the name of one of the entity’s properties and may contain several of them. The main reason for this is that the properties are actually the columns or fields of the entity in question. In addition to the name, these include data types and other optional information, such as its maximum length (in the case of text) or whether it can contain null values.
On the other hand, with the NavigationProperty we define links to other entities. For example, as in the case of BestFriend being a link to another instance of type Person, or Friends being a link to a collection of entities of type Person. To obtain the data from the Person table, we are going to add the word People to the root (the reason for this name change will be explained later):
GET basePath/TripPinRESTierService/People{
"@odata.context": "https://services.odata.org/TripPinRESTierService/(S(3mslpb2bc0k5ufk24olpghzx))/$metadata#People",
"value": [
{
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
{"Person2"},
{"Person3"},
"...",
{"PersonN"}
]
}As you can see, the API has responded with a message with two main elements: the OData context and the value of the response. The context is nothing more than the location of the data schema in the metadata. The response value is the data itself, which in this case contains all the Person entries in the data source. Using the entity key in parentheses we can request individual Person entities, for example using People(‘russellwhyte’) to get just the Person with UserName == russellwhyte.
How to display the NavigationProperty
So, as we can see, the values for all the entity’s Property, which are its fields, do not appear at all with respect to the NavigationProperty. This is because by default these are not shown, as doing so could bring a lot of problems. To show them we have to use one of the query parameters, the expand, being able to ask for more than one of them separating them by commas. For example:
GET basePath/TripPinRESTierService/People('russellwhyte')?$expand=BestFriend,Friends{
"@odata.context": "https://services.odata.org/TripPinRESTierService/(S(e2canygeoavavzu0nrasd4ov))/$metadata#People(BestFriend(),Friends())/$entity",
"UserName": "scottketchum",
"FirstName": "Scott",
"LastName": "Ketchum",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Scott@example.com"
],
"FavoriteFeature": "Feature1",
"Features": [],
"AddressInfo": [
{
"Address": "2817 Milton Dr.",
"City": {
"Name": "Albuquerque",
"CountryRegion": "United States",
"Region": "NM"
}
}
],
"HomeAddress": null,
"BestFriend": {
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
"Friends": [
{
"UserName": "russellwhyte",
"FirstName": "Russell",
"LastName": "Whyte",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Russell@example.com",
"Russell@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [
"Feature1",
"Feature2"
],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
},
{
"UserName": "ronaldmundy",
"FirstName": "Ronald",
"LastName": "Mundy",
"MiddleName": null,
"Gender": "Male",
"Age": null,
"Emails": [
"Ronald@example.com",
"Ronald@contoso.com"
],
"FavoriteFeature": "Feature1",
"Features": [],
"AddressInfo": [
{
"Address": "187 Suffolk Ln.",
"City": {
"Name": "Boise",
"CountryRegion": "United States",
"Region": "ID"
}
}
],
"HomeAddress": null
}
]
}Here we notice two things. Firstly, we see that the message no longer has the ‘value’ field, but directly adds the @odata.context field to the entity properties. This is because now we are no longer asking for a collection of entities, but a single element with the parentheses. Secondly, we can see that this time the fields BestFriend and Friends do appear in the response, being entities or collections of type Person, but without their NavigationProperties.
This example gives us a very clear clue of the danger of always showing the NavigationProperties. I am a friend of my friend who is my friend and I am his friend … We would enter an infinite loop of mutual references.
Property and NavigationProperty data types
To close this block, let’s now talk about the data types available for Property and NavigationProperty. The standard defines the most common in the programming world as String, and they are named using the acronym EDM (Entity Data Model) before the name (Edm.String). However, it is very important to always check the type compatibility between our data sources and OData types. In addition to the standard ones, OData allows you to define your own types, including EntityTypes, as we have already seen with the NavegationProperty, EnumTypes and ComplexTypes. In the next part of the file we see some examples.
<EnumType Name="PersonGender">
<Member Name="Male" Value="0" />
<Member Name="Female" Value="1" />
<Member Name="Unknown" Value="2" />
</EnumType>
<ComplexType Name="Location">
<Property Name="Address" Type="Edm.String" />
<Property Name="City" Type="Trippin.City" />
</ComplexType>
<ComplexType Name="City">
<Property Name="Name" Type="Edm.String" />
<Property Name="CountryRegion" Type="Edm.String" />
<Property Name="Region" Type="Edm.String" />
</ComplexType>
<EntityType Name="Employee" BaseType="Trippin.Person">
<Property Name="Cost" Type="Edm.Int64" Nullable="false" />
<NavigationProperty Name="Peers" Type="Collection(Trippin.Person)" />
</EntityType>EnumTypes, as you can see, are used to define enumerations. They are very useful for fields with unique and limited values. On the other hand, ComplexTypes are very similar to EntityTypes with the difference that they do not have any Key and that they cannot be referenced by themselves, they always have to be a type of a field of an EntityType. In the example above, we can see how even a ComplexType like City can be used inside another ComplexType like Location. We can also see how new EntityTypes can be defined using other EntityTypes as a base, for example the case of Employee.
What are actions and functions
The OData standard relies on CRUD (Create, Read, Update, Delete) operations to interact with data sources, but it also allows the capabilities of data sources to be extended through the use of Actions and Functions, defined and integrated into the data source.
The main difference between actions and functions is that actions can modify the state of the data source, such as updating a record, while functions cannot. Also, given a set of parameters and a database state, functions will always return the same results, whereas actions do not have to.
Finally, functions are invoked using the HTTP GET method, while actions are invoked using the POST method. This is because, in general, functions are used to read data, while actions can insert, update or delete data. Let’s look at some examples of both in the metadata:
<Function Name="GetPersonWithMostFriends">
<ReturnType Type="Trippin.Person" />
</Function>
<Function Name="GetFriendsTrips" IsBound="true">
<Parameter Name="person" Type="Trippin.Person" />
<Parameter Name="userName" Type="Edm.String" Nullable="false" Unicode="false" />
<ReturnType Type="Collection(Trippin.Trip)" />
</Function>
<Function Name="GetInvolvedPeople" IsBound="true">
<Parameter Name="trip" Type="Trippin.Trip" />
<ReturnType Type="Collection(Trippin.Person)" />
</Function>
<Action Name="ResetDataSource" />
<Action Name="UpdateLastName" IsBound="true">
<Parameter Name="person" Type="Trippin.Person" />
<Parameter Name="lastName" Type="Edm.String" Nullable="false" Unicode="false" />
<ReturnType Type="Edm.Boolean" Nullable="false" />
</Action>
<Action Name="ShareTrip" IsBound="true">
<Parameter Name="personInstance" Type="Trippin.Person" />
<Parameter Name="userName" Type="Edm.String" Nullable="false" Unicode="false" />
<Parameter Name="tripId" Type="Edm.Int32" Nullable="false" />
</Action>In this extract from the metadata file we see three functions and three actions. In the two types of methods we have defined the name and type of the parameters, as well as the types for the data they return. All this if indeed these methods need parameters or answer something in the form of a message.
For example, the action ResetDataSource presumably returns the database to a base state, without returning any message or requiring any parameter. Both actions and functions can have the boolean tag IsBound defined, which by default will be false. This tag indicates whether the function or action is bound to a particular entity type, which the standard always associates with the type of the first parameter of the function.
In addition, to avoid ambiguities when invoking a function or action, there is also the EntitySetPath tag, which points to the entity on which we can invoke the methods. To call some of these functions and actions we would use URLs of this type:
GET basePath/TripPinRESTierService/GetPersonWithMostFriends
GET basePath/TripPinRESTierService/People('russellwhyte')/GetFriendsTrips(userName='scottketchum')
POST basePath/Trippin.svc/People(UserName='russellwhyte')/UpdateLastName
Content-Type: application/json
{
"lastName": "Smith"
}Entity container in OData metadata
As a final point in our journey through OData metadata, let’s talk about the entity container. This is where the magic really happens and all the pieces of the puzzle fit together.
Containers have a name, in this case Container, and within them are organised and defined the resources that are actually accessible from the outside with the API. So anything that is not in there, however defined in the CSDL file, does not exist from the point of view of the service consumer. Let’s look at the EntityContainer of this API:
<EntityContainer Name="Container">
<EntitySet Name="People" EntityType="Trippin.Person">
<NavigationPropertyBinding Path="BestFriend" Target="People" />
<NavigationPropertyBinding Path="Friends" Target="People" />
</EntitySet>
<EntitySet Name="Airlines" EntityType="Trippin.Airline">
<Annotation Term="Org.OData.Core.V1.OptimisticConcurrency">
<Collection>
<PropertyPath>Name</PropertyPath>
</Collection>
</Annotation>
</EntitySet>
<EntitySet Name="Airports" EntityType="Trippin.Airport" />
<Singleton Name="Me" Type="Trippin.Person">
<NavigationPropertyBinding Path="BestFriend" Target="People" />
<NavigationPropertyBinding Path="Friends" Target="People" />
</Singleton>
<FunctionImport Name="GetPersonWithMostFriends" Function="Trippin.GetPersonWithMostFriends" EntitySet="People" />
<FunctionImport Name="GetNearestAirport" Function="Trippin.GetNearestAirport" EntitySet="Airports" />
<ActionImport Name="ResetDataSource" Action="Trippin.ResetDataSource" />
</EntityContainer>EntitySets are nothing more than the collections of entities of a particular type available to the API. EntitySets are analogous to tables in a relational database. As we see in this example, the first EntitySet is People, just as we accessed entities of type Person at the beginning of this article. Endpoints always refer to EntitySets, not EntityTypes. We can see that People is a collection of Person and, in addition, we can see how it is necessary to define also the names and types of the NavigationProperties.
Singletons, as the name suggests, are individual instances of entities within the server. They are defined separately because they are not collections of entities.
Finally, and in the same way as everything else, for a function or action to be available from the outside, it must be included in the entity container with a reference to its name and the EntitySet it uses.
Conclusion
In this article we have taken a first look at the OData v4 standard, introducing the CSDL metadata file in OData v4 and what information we can get from it. We talked about the different entities, types and properties, the functions and actions, and finally, we discussed how all these pieces fit together in the entity container to expose it to the outside world.
In the next installment dedicated to OData v4, we will talk about query syntax, query parameters, and other details that have been left out of this article.
Until next time, and happy data management!
If you found this article interesting, visit the Data Engineering category of our blog to see posts similar to this one and share it in networks with all your contacts. Don’t forget to mention us to let us know your opinion @Damavisstudio. See you soon!
