Introduction
This introduction gives an overview of the whitepaper from the point of view of the context, goal, use cases, value proposition, EDI structure, methodology and database.
Context
Currently, TIC encompasses mainly Terminal-related technical data. Operation-related data is not available to the required extent. This data is available separately in the EDIFACT format. The objective is to support the End-To-End operational process.
The review of EDI is done to ensure that the TIC4.0 Model can hold all transferred information.
Nowadays, EDIFACT is the established method to exchange information between terminals. For example, operator orders informing about the expected arrival and departure of cargo along with the associated visits & means of transport. These are used in the TOS for operational planning on the terminal e.g. berth planning, stacking or (un)loading.
Challenges
The main challenge is to obtain valuable information or create useful automated “triggers” as EDI lacks integration with TIC.
Advantage
With the integration of EDI data in TIC4.0 a more holistic view of the terminal performance can be archieved. A positive side effect is to set the EDI data in the context of the TIC4.0 which makes them easier to understand. It offers the capability to fuse data with container and vessel operators, on the one hand, and local terminal operations, maintenance, and yard optimization, on the other.
TIC4.0 will provide the metadata from EDI to create Data Value. It offers an overview of how to translate and incorporate EDI data, from any source (EDI message, TOS, or whatever) to a TIC message, register or data lake (centralized data repository).
Goal
The EDIFACT data will be translated to the TIC4.0 Data standard/model via the provision of definitions, data formats, data models and examples to easily implement the standard.
Potential use cases
Terminal Operators: TOS AUGMENTATION
VALUE PROPOSITION: Empower what the TOS does by adding different data sources to TOS capabilities using the same language to reduce complexity on integration and adoption. Enable the data before it has been processed into the TOS or validated to anticipate the actions.
Automated workflow checklist
COPRAR received (3 / 8)
MOVINS
BAPLIE
Gang assignment
OUTBOUND BAPLIE (or any other EDI):
Automatically send to other stakeholders in TIC4.0
Stakeholders will receive the BAPLIE and
Vehicle booking system prediction
CODECO vs COPINO trends (which was the planned vs executed?)
COPRAR-D
BAPLIE + MOVINS:
Trigger alarms to users in order to adapt QC sequences to tackle those vessel bays with higher stowage plans during low tide
Suggest actions to users in order to nominate the required personnel to perform special operations such as OOG containers that need to be handled with chains / slings
To know what is going to be discharged / loaded on vessel calls
Make use of advanced capabilities to make predictions from data trends and determine influencing factors.
For instance, to determine when containers are going to depart from the terminal (vessel, gate…), dwell times according to FDEST, vessel rollouts, gate out date …
Integrate different systems for use on emulation / simulation environments using the same standard
Global Terminal Operators
VALUE PROPOSITION: Reduce the integration efforts across terminals
Combine data from different terminals (and different TOS) using the same standard. In case they are managing terminals around the world and the terminals currently use different standards
X12 (ANSI)
EDIFACT
TOS (Navis, Cyberlogitec, TSB, RBS …)
Process orchestration and workflow management across different terminals in the portfolio to include different data sources using the same format
Facilitate easier integration with minimized effort to make use of data which is combined from different sources
EDI (COARRI, BAPLIE, MOVINS, COPRAR, CODECO, COPINO …)
TOS (N4, CATOS, OPUS …)
Vessel calls information
Yard Inventory
Verified load / discharge list
M&R
Equipment (availability …)
Weather API
Added value:
Trigger alarms prior to processing in TOS for critical discrepancies between EDI files and other sources such as Yard Inventory
For instance:
IMO 1 container to be received by Gate (info from CODECO) suggest actions to users prior to receiving the container
Allow better operations preparation in case a certain number of empty containers are ordered to be loaded on vessel
Combine information from different sources
For instance: predict yard fluctuations for the next week, combining different data sources, depending on if EDI are received and processed on TOS
Historic -> Estimations -> EDI -> Definitive lists
Agencies
VALUE PROPOSITION: Be connected to different data sources from stakeholders (as terminals, central planners…) using the same standard. For instance: Send EDIs as converted to TIC to terminals and shipping line (COPRARs and Bookings) and receive validated load and discharge list in TIC (For which there are not currently EDI standard)
Modernize interfacing EDI
VALUE PROPOSITION: Intermediate step to modernize and reduce the complexity of the EDI. TIC4.0 global adoption will take time. In the meantime, a translator is needed between EDI and TIC to enable all terminals and systems integration using the same standard. Next step will be to communicate directly using TIC and not EDI anymore.
EDI structure
The base of the Task Force review and conversion is the Ship Message Design Group (SMDG) Standard which is published by the SMDG group, see EDIFACT MIGs – SMDG e.V. .
The conversion focuses on established messages as per the criteria of SMDG.
The following message types will be reflected in the conversion:
Type | Name | Short description |
|---|---|---|
Ship Planning | BAPLIE | Bayplan |
MOVINS | Stowage Instruction | |
Container Messages | COPRAR | Vessel Discharge / Loading Order |
COARRI | Vessel Discharge / Loading Confirmation | |
COPARN | Full-in Booking Order / Empty-out Release | |
CODECO | Gate in/out confirmation | |
COREOR | Full-out Release | |
COPINO | Pre-arrival Notice | |
Vessel Schedule | IFTSAI | Used for transmission of vessel schedule |
VESDEP | Vessel Departure Message | |
Terminal Performance | TPFREP | Terminal Performance Report sent from a terminal to a carrier. |
VERMAS | VERMAS | Used to report VGM (Verified Gross Mass) |
As a starting point, the smaller EDI message COARRI was chosen to gain experience with the structure and find a good way to review and document the results.
Structure of the EDI File:
The EDI Files follow the same structure and definition of data.
More details about this can be found in the documentation for each message type at EDIFACT MIGs – SMDG e.V. There are several tools on the internet available to aid the user in reading and understanding the EDI messages.
The following explanation is given to help understand the way in which the transfer from EDI to TIC4.0 can be made. It does not provide a complete explanation of the EDIFACT structure and logic.
EDIFACT messages are split into segment groups and segments. The segment itself holds TAGs and values.
The information in the EDI files is classified with the TAGs which identify the kind of content.
Each message has some segments to identify the general message type and give some control information.
Below is the general structure of the files with the TAGs which always have to exist in a file (bold entries):
UNH Service segment to start and uniquely identify a message |
BGM A segment to indicate the beginning of a message and to transmit an identifying number and the further specification of the message type. |
Message Body is separated in Segments and Groups dependent on the kind of message |
CNT A segment to specify the number of containers in the message, explicitly given by the sender. |
UNT A service segment ending a message, giving the total number of segments in the message (including the UNH & UNT) and the control reference number of the message. |
Segments
The way to read and translate the EDIFACT files is explained with the help of some examples.
General information to the structure of the segments:
Each Segment ends with a ‘.
+ is used as a separator between the elements
In the case of composite elements, “:” is used as a separator.
To read a segment we analyze from left to right the content between two +.
Example A:
DTM+132:202302221600:203'
DTM - The TAG gives the information that the segment will contain Date and Time information
132:202302221600:203
132 - code for arrival time.
202302221600 – confirmed date and time of the arrival
203 – code which specifies the format of the date and time information: JJJJMMTThhmm
Extract from the SMDG Standard document related to this example line:
The use of code lists:
References and standardised information are linked with codes and tracked in lists.
These lists can be used in different contexts, but not in all contexts are all values allowed.
In the above example, the 132 is an entry of a code list which means that the following information will be the arrival time.
In the format description of the related EDI message the number of the code list and the allowed values are listed. The first value in the example which is part of such a list is the 132 as code of the arrival date / time.
As you can find the extract below 2005 is the id of the list. 132, 133, 178 and 186 are the allowed values.
The next example in this line is the 203. This is an entry from the code list 2379. In this case only the value 203 is allowed.
Example B
EQD+CN+NYKU4116702+45G1:102:5++3+5'
EQD – TAG for Equipment Details
CN – Kind of Equipment and Element of List 8053 (Allowed values are: BB, BPP, CH, CN - Container, RG, SQ, TE)
NYKU4116702 - Equipment Identifier
45G1:102:5
45G1 - Size and Type Description Code
102 element of List 1131 identification code (allowed value 102)
5 element of list 3055 (responsible agency – allowed value 5)
+ + - missing, not mandatory information – equipment supplier code
3 - equipment status code – element of list 8249 (allowed values are 1,2,3,5,6,7,8,14)
5 - full or empty indicator code – element of list 8169 (allowed values 4, 5, 6)
Translation Methodology
The review of the EDI messages and translation was done by going through the EDI standard segment-wise. For each segment we considered the possible values transferred and allowed in each TAG. These are either translated in existing TIC4.0 fields or new fields were defined.
To explain the definition which was done there is always one example with concrete values translated.
The table used contains several columns to define the ‘to-be-mapped’ content and the TIC Data Mapping fields.
Column | Name | Content |
|---|---|---|
A | ### | counter / id |
B | Possible Code | Allowed Codes according to SMDG |
C | EDI SMDG Data Element | Name of the data element |
D | EDI SMDG Component Element | Name of the data component element (like list number) |
E | EDI SMDG Data name | SMDG Name of the data |
F | “Human Name” | Explanation which data are kept in this field |
G | TIC Data Mapping | Mapping in the TIC Data structure |
In the following, a similar example like example A above is now used to explain how the translation is done and how to read the attached EXCEL Sheet.
Example 1:
DTM+133:200211131700:203'
Extract from the EXCEL Sheet with the translation of this example:
B | C | D | E | F | G |
C507 | DATA / TIME / PERIOD | ||||
132 - Arrival date/time, estimated | 2005 | Date or time or period function code qualifier | Date / Time Information of the Transport | tos|@|carriervisit|@|departure|@|mooring|@|firstline|time|planned|timestamp: 2002-11-13T0917:00.000Z | |
2380 | Date or time or period value | Arrival/departure date/time | |||
203 - CCYYMMDDHHMM | 2379 | Date or time or period format code | COARRI is CCYYMMDDHHMM format ant TIC4.0 is always in ISO8601 format | ||
DTM is the TAG for the SMDG data element C507 - Date / Time Period. This explains the following information and doesn’t require fields in the TIC4.0 wording.
In the example DTM is followed by a +, which is the separator between TAG and information transferred in this line.
DTM+133:200211131700:203'
The value 133 is an entry of the SMDG component element 2005 (Date or time or period function code qualifier).
We have 4 different allowed values which describe “different meanings” for the following date. This means that they have to go to different fields in the TIC 4.0 format.
132 - Arrival date/time, estimated | tos|@|carriervisit|@|arrival|@mooring|@|firstline|time|planned|timestamp: 2380 if 2005=132 |
133 - Departure date/time, estimated | tos|@|carriervisit|@|departure|@|mooring|@|firstline|time|planned|timestamp: 2380 if 2005=133 |
178 - Arrival date/time, actual | tos|@|carriervisit|@|arrival|@|mooring|@|firstline|time|actual|timestamp: 2380 if 2005=178 |
186 - Departure date/time, actual | tos|@|carriervisit|@|departure|@|mooring|@|firstline|time|actual|timestamp: 2380 if 2005=186 |
The component element 2380 holds the date and time: 200211131700.
The component element 2379 holds the format description of the component element 2308: CCYYMMDDHHMM.
Example 2
This example explains the use of the lists from smdg in the TIC4.0 database related to the EDI translation
NAD+CF+MAE:172:20'
B | C | D | E | F | G |
CA Carrier | 3035 | PARTY FUNCTION CODE QUALIFIER | tos|@|cargo|@|responsableparty|metadata|output|actual|#qualifier#20|partyfunctioncode : CF | ||
C082 | PARTY IDENTIFICATION DETAILS | ||||
3039 | Party Identifier | Company Code | tos|@|cargo|@|responsableparty|metadata|output|actual|#qualifier#20|output|actual|value : MAE | ||
160 Party identification | 1131 | Code list identification code | Kind of Party | tos|@|cargo|@|responsableparty|metadata|output|actual|#qualifier#20|codelistidentificationcode : 172 | |
20 BIC (Bureau International des Containeurs) | 3055 | Code list responsible agency code | tos|@|cargo|@|responsableparty|metadata|output|actual|#qualifier#20|qualifier : 20 |
NAD is the SMDG TAG for name and address.
The following information is relevant after the +, which specifies the date information in more detail.
CF is an entry of the smdg data element 3035 which has a fix list of allowed values:
CA Carrier
CF Container operator/lessee
HR Shipping line service
MR Message recipient
MS Document/message issuer/sender
The definition in the TIC data model is done as follows. There are metadata defined for it. The metadata partyfunctioncode holds the number of the list. :he metadata partyfunctionname holds the list with the possible values.
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055|partyfunctioncode : 3035
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055|partyfunctionname : 3035 (see list)
example: tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#20|partyfunctioncode : CF
MAE:172:20
This is the smdg data element C082 which holds the party identification details.
MAE is the data value. After the :'s it is described what exactly this data value describes. As each filed in smdg this one has a number: 3039
This is translated as follows:
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#20|output|actual|value : MAE
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055|output|actual|value : 3039
172 is an entry of list 1131 and describes the kind of the party.
The allowed values in the list are
172 Carrier
160 Party Identification
This is translated similar to the example before:
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#20|codelistidentificationcode : 172
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#20|codelistidentificationname : Carriers
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055codelistidentificationcode : 3039
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055codelistidentificationname : 172= Carriers/160 Party identification
20 is an entry of the list 3055 and describes the code list standards from which the data values were taken.
The allowed values in the list are
20 BIC (Bureau International des Containeurs)
87 Assigned by carrier
166 US, National Motor Freight Classification Association
184 AU, ACOS (Australian Chamber of Shipping)
SMD SMDG (Shipplanning Message Development Group)
ZZZ Mutually defined
This is translated as follows:
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#20|qualifier : 20
tos|@|cargo|@|responsibleparty|metadata|output.actual|#qualifier#20|codelistresponisbleagencyname : BIC
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055|qualifier : 3055
tos|@|cargo|@|responsibleparty|metadata|output|actual|#qualifier#3055#|codelistresponisbleagencyname : 3055 see list
DATA BASE
The backend of this code is public available at