How to Successfully Integrate in Data Exchange Projects with Business Partners

There is no doubt that EDI (the Electronic Data Interchange) offers numerous opportunities for cost, time, efficiency and safety optimization. However, the questions we often hear are: Is EDI integrated with SAP or another ERP system? Why would I even need a classic EDI solution when the API method is already there? This article provides clear answers, so you can run the integration process successfully in your data exchange projects.

After the decision to start fully digitized data exchange in your company, you can take one of two approaches: 

• Classical and direct EDI integration via communication channels such as AS2, X.400, SFTP, FTPS and OFTP
• Introduction of an API (application programming interface)

The right model for you will depend on your needs, but in either case your data exchange solution provider must have the technical capabilities to implement it and should not be bound by a certain model. Both models have advantages and disadvantages. Each works best in different circumstances, as explained below.

What do we understand by EDI integration and different protocols?

EDI integration has been around for more than 50 years. It is the  process of introducing comprehensive EDI workflow between business partners. Before deciding which protocol best suits your business needs, you need to answer several key questions, including:

• Business partners: with whom are you exchanging documents via EDI?
• Which IT system needs to send, receive and process EDI data, for example, ERP, WMS, etc.?
• Will you exchange data between web-based systems or mobile applications?
• What messages or documents need to be exchanged between you and your partners?
• What EDI formats do your messages need to follow?
• Which technologies are you going to use to transmit your messages?

Having this information, the next step is to establish a communication channel connection. Usually, you choose between the most popular ones mentioned above: AS2, X.400, SFTP, FTPS, and OFTP. What are the differences between them?

AS2

AS2 is probably the most popular method for data shipment (especially EDI data), known for its security and reliability over the Internet. It was created in 2002 by the IETF to replace the AS1 they created in the early 1990s. AS2 usually involves two computers: the client’s one and a server connected point to point through the web. AS2 creates an envelope for the EDI data, allowing information to be sent using digital certificates and encryption. What is important, is that this method requires certificates for signing and encryption on both sides. A feature unique to AS2 is that it sends a message delivery notification as confirmation of receipt.

Walmart was the first major retailer to make its suppliers use the AS2 protocol instead of relying on dial-up modems for ordering goods. Amazon, Target, Lowe's, Bed, Bath & Beyond and thousands of others followed. These companies have already proved they can cut costs and achieve near real-time EDI communication with direct AS2 EDI, bypassing expensive, legacy value-added networks (VANs).

X.400

X.400 is an old solution. It is basically “‘email on steroids”. The addressing is extremely structured and was used as an alternative to a more common email protocol called Simple Mail Transfer Protocol (SMTP). X.400 is commonly used only in the EDI world. 

Take a look at the comparison. An email address in SMTP looks like this: john.smith@comarch.com.pl. An equivalent in X.400 would be: G=John, S=Smith, O=COM, OU=Comarch and C=PL. An X.400 address consists of several elements: C: country name, ADMD: administration management domain, PRMD: private management domain, O: organization name, OU: organization unit name, G: given name, I: initials, S: surname.

X.400 establishes a mailbox similar to an email on the client and EDI provider sides. An X.400 address and application are required. Statuses for sending, receiving and delivering are possible. An important thing is that document exchange via the X.400 protocol is associated with additional costs for the EDI provider, so you should avoid connecting via X.400, or pass this cost on to the client.

FTP and SFTP, FTPS and OFTP

FTP stands for File Transfer Protocol. It is a standard network protocol used for the transfer of computer files between a client and server on a computer network. FTP is built on client server model architecture, using separate control and data connections between the client and the server. FTP users may authenticate themselves with a clear text sign in protocol, normally in the form of a username and a password. FTP generates no statuses except the internal ones for setting messages of the FTP folder.

FTPS (FTP over SSL/TLS) stands for File Transfer Protocol Secure. It uses secure encryption. A session key protects the data being transported. Messages are encrypted with a session-specific key (TLS handshake). Once established, all messages exchanged between the client and server are encrypted.

With FTPS, the user may authenticate the sender's server identity by validating the trustworthiness of the server's certificate running several checks. Most notably, this allows the user to check whether the certificate was issued by a trusted certificate authority. The server authenticates the client using username and password over a secure channel.

Focusing on other types of channels, SFTP (FTP over SSH) stands for Secure File Transfer Protocol. SFTP requires that communication between the client and server is fully secured. As a result, SFTP makes it easier for IT administrators to enforce security best practices within an organization by standardizing all file transfers.

OFTP stands for ODETTE File Transfer Protocol. It is a protocol used for EDI. The first version of this protocol was created in 1986. OFTP is popular across the European automotive industry. OFTP2 can work point to point or indirectly via a VAN (value-added network). A single OFTP2 entity can make and receive calls, and exchange files in both directions. This means that OFTP2 can work in push or pull mode.

Additionally, OFTP2 can encrypt and digitally sign message data, and request signed receipts, while also offering high levels of data compression. All of these services are available when using OFTP2 over TCP/IP, X.25/ISDN or native X.25. When used over a TCP/IP network, such as the Internet, additional session-level security is available by using OFTP 2 over TLS (Transport Layer Security).

Web Service

Another channel is Web Services, which is a way of communicating between two devices: client and server. It uses a standardized XML messaging system via the Internet in HTTP. A client uses Web Services by sending an XML message. Then, they wait for a real-time XML response from the server. Establishing web services does not require more effort than the other channels.

Since all messages are in XML, Web Services are an efficient method for exchanging data electronically. Thus, you are not bound to any system or application.

Having in mind the basics of the classic EDI, the next step is to understand what an API is, and how it works.

What do we understand by “introducing an API”, and how is this related to the data exchange process?

API stands for application programming interface. It is a software intermediary that allows two applications to communicate. It was developed to allow applications to reach data and interact with external software components or operating systems. An API delivers a user response to a system and sends the system's response back to a user.

But, standardization is a real problem. This makes it difficult to integrate different APIs. To solve this problem and make APIs easier to integrate, many different standards have been developed, two of which dominate. These are SOAP, which stands for Simple Object Access Protocol, and REST – Representational State Transfer.

SOAP API

SOAP is a standard communication protocol system that allows processes using different operating systems to communicate by HTTP and XML. SOAP based APIs exist to create, recover, update, and delete records such as accounts, passwords or custom objects.

This offers over 20 different kinds of calls that make it easy for API developers to maintain their accounts, perform accurate searches and much more. Therefore, it can be used with all those languages that support web services.

SOAP APIs take advantage of making web-based protocols such as HTTP and XML, which are already operating. This is why developers can easily manipulate web services and get responses without being concerned about languages and platforms.

REST API

Representational State Transfer is a standard for designing APIs where CRUD (CREATE, READ, UPDATE, DELETE) actions correspond to POST, GET, PUT and DELETE methods. REST does not require WDSL or UDDI, but is based on several principles, such as:

• Uniform interface for communication between client and server. This can be described by four internal rules:
  • First, the interface is based on resources – each resource has its own unique URI, but the way it is represented can be different and is independent of the resource itself. For example, the server does not send the data as read directly from the database, but as JSON, for instance. 
  • Second, the manipulation of resources is carried out through representation. After receiving the representation of the resource from the server, the client (if it has the rights) may modify or delete the resource. 
  • Third, the server responses to the client are self-describing. Each server response contains information on how to handle it. 
  • Fourth, HATEOAS (hypermedia as the Engine of Application State) means there are relationships between resources, and the client should be able to navigate between them using what is called hypermedia.
• Client-server. Defines a clear separation between server-side and client-side logic, so that editing one does not affect the other.
• Stateless. This rule states that the server should not store information about the client state. Each request should be complete in terms of the information needed to process it.
• Cacheable. API should support caching to improve performance. As a rule, caching is available for read operations, while it is not recommended for data modification operations.
• Layered system. When designing a system, it is important to pay attention to the fact that the client sending a query should get an answer from the server being queried. This should be done without knowing what is happening "underneath". For example, in a situation where a server queries other external servers such as Google, etc.
• Code on demand. This is an optional rule that assumes the ability to send code that we can execute on the client side (often JavaScript).
  
  

EDI vs. API: the most common challenges

In the data exchange environment via APIs, you access software allowing real-time communication with business partners. This establishes a connection between different systems, but does not require any user intervention. API enables a real-time connection, constant access to the inventory, and the possibility to send updated data. Its effectiveness depends on the removal of the time needed to request and send information.

So, if you introduce an API, prepare yourself for the fact that new data formats may not always supported. There is no requirement for  information about, for example, identifying users or products (such information may include GTIN or GLN). Businesses introducing APIs may face problems when adding new partners, which often demands changes to ERP systems. No standard information may be contained in this API to exchange data (which is a cost driver).

Another question is: should you connect partners with their own APIs? There is no data standard implemented or even recommended in the API creation process. This introduces another cost driver in terms of switching to multiple partners via API communication. If an API is not set correctly, sending a simple document may demand additional effort. An example might be asking for an order number or header data. 

The other thing is that API might not cover the entire supply chain communication. More business partners equals more systems to be combined with each other. Imagine how complicated this may be, especially if you have thousands of business partners.

So, combining different APIs and making them compatible or able to communicate requires efforts which translate to money and time. For this reason, the use of APIs is not recommended in most data exchange projects.

EDI has a number of adantages in comparison to an API. The biggest one is the standard of document transfer.  Unlike with API, you do not have to be prepared for new data formats that are not always covered by your API. Adding new business partners is not difficult. You just choose a communication channel, type in details for identification of your business partner required by the chosen channel, and… that’s it. Classic EDI requires usually no changes in the ERP system of business partners which are designed to be connected via the protocols explained above.

Another thing is that classic EDI is not only secure (due to data encryption and certificates), but it is also simple, thanks to an exhaustive number of data standards such as UN/EDIFACT, ANSI ASC X12, TRADACOMS, VDA and UBL. There is no place for surprises during the implementation phase, meaning no invalid details, formats or fields to be filled out. The protocols ensure the delivery of the sent data by sending statuses which are checkable. In other words, simplicity replaces the complexity of an API. This is reflected by another fact – that classic EDI has one communication method with your service provider and one mapping set for each transaction. Although an API solution enjoys real-time communication, EDI might be slower, but still the speed of data shipment is high (taking just a couple of seconds). Touching the topic of data traffic, EDI represents another important asset – it allows traffic of high data volume. Sending and receiving thousands of documents is not a problem at all.

Summary: and the winner is…

Although API is an advanced tool with multiple possibilities, in the field of data exchange and integrating business partners it is rather hard to see it as the number one choice. We expect that classic EDI is going to lead for the coming years, as the most efficient and universal solution. In the IT services and solutions landscape the niche of data exchange in the form of business-related documents is, and will remain, dominated by classic EDI.

About the Author

Michał Kacperek
Business Solutions Consultant, Comarch