Performance

Reference

Title

EBA requirement

OBL calculation guidelines

OBL recommended benchmark

EBA Guideline 2.3 (a)

PISP response time

…the ASPSP should define, at a minimum, the following KPIs for the performance of the dedicated interface:

a) the daily average time (in milliseconds) taken, per request, for the ASPSP to provide the payment initiation service provider (PISP) with all the information requested in accordance with Article 66(4)(b) of PSD2 and Article 36(1)(b) of the RTS;

OBL recommends that the following elements are reported as an average response time in ms for each day:

1. Average PIS response time = Avg Ta + Avg Tb + Avg Td + Avg Te + Avg Tg      (Mandatory)
2. Average PSU authentication time = Avg Tc = (Tc) / (Vc)                      (Optional) 
3. Average PIS Y/N confirmation = Avg Tf = (Tf) / (Vf)                              (Separate Reporting)
4. Average PIS status response = Avg Th = (Th) / (Vh)                             (Separate Reporting)

The following API endpoints should be included when calculating PISP response times:

Avg Ta & Avg Te: Average Time Periods a & e

The following OIDC endpoints should be reported to be included in the calculation for the response time of time periods Ta & Te:

• OIDC endpoints for token IDs

Where an ASPSP cannot separate calls to the OIDC token endpoint-based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents.

The average response time of all endpoints in time periods Ta & Te shown above should be calculated as follows:

Avg Ta = Avg Te = (Ta & Te) / (Va & e)

where Va & e is the total volume of calls made to the end-points listed for time periods Ta & Te

Avg Tb: Average Time Period b

The following API endpoints relevant to the payment consent staging should be included in the calculation for the response time of time period Tb:

• POST /domestic-payment-consents
• POST /domestic-scheduled-payment-consents
• POST /domestic-standing-order-consents
• POST /international-payment-consents
• POST /international-scheduled-payment-consents
• POST /international-standing-order-consents

The average response time of all endpoints shown above should be calculated as follows:

Avg Tb = (Tb) / (Vb)

where, Vb is the total volume of calls made to the end-points listed for time period Tb

Please note that in the case of payment staging API endpoints from the above list (i.e. POST endpoints), the response time must include “the time it takes for any checks that the ASPSP may choose to make on the authorisation/registration of TPPs”. This may include validation of access token, validation of the request payload, authentication/validation of the request etc.

Avg Tc: Average Time Period c

PSU Authentication time (PSU involvement) (OPTIONAL)

This is the total time taken for the PSU to complete the authentication process. For the avoidance of doubt, this is the time measured in seconds from the point in time the PSU, after being redirected to the authentication screen of the ASPSP Authorisation Server, submits the credentials for authentication, completes authentication and any other actions such as selecting account(s), until control is passed back to the ASPSP to start the process of generating the authorisation code. 

The average response time of all endpoints shown above should be calculated as follows:

Avg Tc = (Tc) / (Vc)

where, Vc is the total volume of PSU authentications that generated the total response time Tc

Avg Td: Average Time Period d

The following is not an OIDC endpoint as such but it is an entry to be used for measuring the time period (d) of the authorisation code generation by ASPSPs:

• Generated OIDC Authorisation code

Where an ASPSP cannot separate calls to the OIDC token endpoint-based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents.

The average response time of all endpoints shown above should be calculated as follows:

Avg Td = (Td) / (Vd)

where, Vd is the total volume of calls made to the end-points listed for time period Td

 

Avg Tf: Average Time Period f

The following API endpoints relevant to the payment consent staging should be included in the calculation for the response time of time period Tf:

• GET /domestic-payment-consents/{ConsentId}/funds-confirmation
• GET /international-payment-consents/{ConsentId}/funds-confirmation
• GET /international-scheduled-payment-consents/{ConsentId}/funds-confirmation

The average response time of all endpoints shown above should be calculated as follows:

Avg Tf = (Tf) / (Vf)

where, Vf is the total volume of calls made to the end-points listed for time period Tf

Avg Tg: Average Time Period g

The following API endpoints relevant to the payment consent staging should be included in the calculation for the response time of time period Tg

• POST /domestic-payments
• POST /domestic-scheduled-payments
• POST /domestic-standing-orders
• POST /international-payments
• POST /international-scheduled-payments
• POST /international-standing-orders

The average response time of all endpoints shown above should be calculated as follows:

Avg Tg = (Tg) / (Vg)

where, Vg is the total volume of calls made to the end-points listed for time period Tg

Avg Th: Average Time Period h

The following API endpoints relevant to the payment consent staging should be included in the calculation for the response time of time period Th:

• GET /domestic-payments/{DomesticPaymentId}
• GET /domestic-payments/{DomesticPaymentId}/payment-details
• GET /domestic-scheduled-payments/{DomesticScheduledPaymentId}
• GET /domestic-scheduled-payments/{DomesticScheduledPaymentId}/payment-details
• GET /domestic-standing-orders/{DomesticStandingOrderId
• GET /domestic-standing-orders/{DomesticStandingOrderId}/payment-details
• GET /international-payments/{InternationalPaymentId}
• GET /international-payments/{InternationalPaymentId}/payment-details
• GET /international-scheduled-payments/{InternationalScheduledPaymentId}
• GET /international-scheduled-payments/{InternationalScheduledPaymentId}/payment-details
• GET /international-standing-orders/{InternationalStandingOrderPaymentId}
• GET /international-standing-orders/{InternationalStandingOrderPaymentId}/payment-details

The average response time of all endpoints shown above should be calculated as follows:

Avg Th = (Th) / (Vh)

where, Vh is the total volume of calls made to the end-points listed for time period Th

The ASPSP’s signed response to the POST will inherently act as proof of receipt of the payment order by the ASPSP, which will enable the TPP to log a reference and the date of this receipt. Both the POST and the GET endpoints contain all information relating to the payment, which, depending on the payment type, should include reference, amount, exchange rate, charges, and status (which may change between POST and any subsequent GET).

The POST endpoints above cater for the requirements of PSD2 Article 66(4)(b), RTS Article 36(1)(b), i.e. for the ASPSP to make the information available to the PISP immediately after receipt of the payment order, and the FCA PSRs Approach Paragraph 17.29, i.e. the provision of all information on the initiation of the payment transaction and all information accessible to the ASPSP regarding the execution of the payment transaction.

The GET endpoints cater for the requirements of the PSRs Approach Paragraph 17.30, i.e. for the ASPSP to provide confirmation to the PISP that payment initiation has been successful, in order to enable the PISP to provide this information to the PSU.

An average TTLB of 750 milliseconds per endpoint response for all but file payments.

Reference

Title

EBA requirement

OBL calculation guidelines

OBL recommended benchmark

EBA Guideline 2.3 (b)

AISP response time

b) the daily average time (in milliseconds) taken, per request, for the ASPSP to provide the account information service provider (AISP) with all the information requested in accordance with Article 36(1)(a) of the RTS;

OBL recommends that the following elements are reported as an average response time in ms for each day:

1. Average AIS response time = (Ta+Tb+Td+Te+Tf+Tg) / (Vg)       (Mandatory)
2. Average PSU authentication time = Tc Avg = (Tc) / (Vc)             (Optional) 

The following API endpoints should be included when calculating AISP response times:

Ta & Te: Time Periods a & e

The following OIDC endpoints should be reported to be included in the total response time:

• OIDC endpoints for token IDs

Where an ASPSP cannot seperate calls to the OIDC token endpoint based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents. 

Tb: Time Period b

The following API endpoints relevant to the account access consent staging should be included when calculating AISP response times:

• POST /account-access-consents

Please note that in the case of account information consent staging API endpoints from the above list (i.e. POST endpoints), the response time must include “the time it takes for any checks that the ASPSP may choose to make on the authorisation/registration of TPPs”. This may include validation of access token, validation of the request payload, authentication/validation of the request etc.

Avg Tc: Average Time Period c

PSU Authentication time (PSU involvement) (OPTIONAL)

This is the total time taken for the PSU to complete the authentication process. For the avoidance of doubt, this is the time measured in seconds from the point in time the PSU, after being redirected to the authentication screen of the ASPSP Authorisation Server, submits the credentials for authentication, completes authentication and any other actions such as selecting account(s), until control is passed back to the ASPSP to start the process of generating the authorisation code. 

The average response time of all endpoints shown above should be calculated as follows:

Avg Tc = (Tc) / (Vc)

where, Vc is the total volume of PSU authentications that generated the total response time Tc

Td: Time Period d

The following is not an OIDC endpoint as such but it is an entry to be used for measuring the time period (d) of the authorisation code generation by ASPSPs.

• Generated OIDC Authorisation code

Where an ASPSP cannot seperate calls to the OIDC token endpoint based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents. 

Tf: Time Period f

• GET /accounts

Tg: Time Period g

• GET /accounts/{AccountId}
• GET /accounts/{AccountId}/balances
• GET /balances
• GET /accounts/{AccountId}/transactions
• GET /transactions
• GET /accounts/{AccountId}/beneficiaries
• GET /beneficiaries
• GET /accounts/{AccountId}/direct-debits
• GET /direct-debits
• GET /accounts/{AccountId}/standing-orders
• GET /standing-orders
• GET /accounts/{AccountId}/product
• GET /products
• GET /accounts/{AccountId}/offers
• GET /offers
• GET /accounts/{AccountId}/party
• GET /party
• GET /accounts/{AccountId}/parties
• GET /accounts/{AccountId}/scheduled-payments
• GET /scheduled-payments
• GET /accounts/{AccountId}/statements
• GET /accounts/{AccountId}/statements/{StatementId}
• GET /accounts/{AccountId}/statements/{StatementId}/file
• GET/accounts/{AccountId}/statements/{StatementId}/transactions
• GET /statements

Vg: Volume of calls g
Total volume of calls made to the end-points listed in Tg.

An average TTLB of 750 milliseconds per response, or per page of results for up to 100 records for larger payloads.

In practice, all but transactions and statements are likely to be small payloads without pagination.

Reference

Title

EBA requirement

OBL calculation guidelines

OBL recommended benchmark

EBA Guideline 2.3 (c)

Confirmation of Funds (CoF) response time (CBPII and PISP)

c) the daily average time (in milliseconds) taken, per request, for the ASPSP to provide the card-based payment instrument issuer (CBPII) or the PISP with a ‘yes/no’ confirmation in accordance with Article 65(3) of PSD2 and Article 36(1)(c) of the RTS;

OBL recommends that the following elements are reported as an average response time in ms for each day:

1. Average CBPII CoF response time = (Ta+Tb+Td+Te+Tf) / (Vf)       (Mandatory)
2. Average PSU authentication time = Avg Tc = (Tc) / (Vc)                 (Optional) 

The following API endpoints should be included when calculating CoF response times for CBPII:

Time Periods a & e

The following OIDC endpoints should be reported to be included in the total response time:

• OIDC endpoints for token IDs

Where an ASPSP cannot seperate calls to the OIDC token endpoint based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents. 

Time Period b

The following API endpoints relevant to the CBPII CoF access consent staging should be included when calculating CBPII response times:

• POST /funds-confirmation-consents

Please note that in the case of CBPII CoF consent staging API endpoints from the above list (i.e. POST endpoints), the response time must include “the time it takes for any checks that the ASPSP may choose to make on the authorisation/registration of TPPs”. This may include validation of access token, validation of the request payload, authentication/validation of the request etc.

Avg Tc: Average Time Period c

PSU Authentication time (PSU involvement) (OPTIONAL)

This is the total time taken for the PSU to complete the authentication process. For the avoidance of doubt, this is the time measured in seconds from the point in time the PSU, after being redirected to the authentication screen of the ASPSP Authorisation Server, submits the credentials for authentication, completes authentication and any other actions such as providing consent, until control is passed back to the ASPSP to start the process of generating the authorisation code. 

The average response time of all endpoints shown above should be calculated as follows:

Avg Tc = (Tc) / (Vc)

where, Vc is the total volume of PSU authentications that generated the total response time Tc

Time Period d

The following is not an OIDC endpoint as such but it is an entry to be used for measuring the time period (d) of the authorisation code generation by ASPSPs.

• Generated OIDC Authorisation code

Where an ASPSP cannot seperate calls to the OIDC token endpoint based it the scope parameter, the ASPSP should apportion the total OIDC calls in the same ratio as AIS, PIS and CBPII consents. 

Time Period f

• POST /funds-confirmations

Vf: Volume of calls f
Total volume of calls made to the end-points listed in Tf.

An average TTLB of 750 milliseconds per response.

This benchmark would not apply to complex corporate models, but rather simple account models only.

Reference

Title

EBA requirement

OBL calculation guidelines

OB recommended benchmark

EBA Guideline 2.3 (d)

Daily error response rate

d) the daily error response rate – calculated as the number of error messages concerning errors attributable to the ASPSP sent by the ASPSP to the PISPs, AISPs and CBPIIs in accordance with Article 36(2) of the RTS per day, divided by the number of requests received by the ASPSP from AISPs, PISPs and CBPIIs in the same day.

It is not possible for ASPSPs to respond to TPPs with an error message where no TLS session has been established. However ASPSPs should still be able to respond, measure and report on errors relating to all OIDC endpoint calls and all functional API calls relating to the Open Banking Standard.

The error response rate should be calculated as the total number of all 5xx HTTP status codes from all API endpoints per day, divided by the total number of TPP API requests received across all of these endpoints in the same day, and expressed as a percentage.

Errors based on 4xx HTTP status codes are largely attributable to TPP or PSU actions or failures, and hence should not be included here.

Cases where 2xx HTTP status codes are returned, but where the data in the response payload is not correct are covered in section Design and Testing.

An average of 0.5% across all endpoints