Short Message Service (SMS) Performance Tests

IMDA currently conducts annual tests on the performance of the Short Message Service (SMS) provided by the mobile operators and the results are published on the IMDA website for public information. The objective of the SMS Performance Test is to measure the end-to-end response time of SMS across local mobile networks.

Note: From 2014, the SMS Performance Test report are published yearly.

What is Measured?

The transmission speed of the SMS involving the three mobile operators, namely, SingTel Mobile, M1 and StarHub Mobile is measured. Specifically, the parameters measured in the test, for both intra-operator SMS (SMS sent and received within a single operator's network) and inter-operator SMS (SMS sent and received across different operators' networks) are: 

Percentage of SMS messages delivered within 15 seconds;

Percentage of SMS messages delivered within 15 minutes; and

Percentage of SMS messages delivered within 12 hours.

Note: IMDA has revised the delivery timeframes of the SMS Performance Tests with effect from 1H 2012. Prior to 1H 2012, the parameters measured in the test were: (i) percentage of SMS messages delivered in less than 20 seconds; (ii) percentage of SMS messages delivered in 1 hour or more; and (iii) percentage of SMS messages not delivered.

Definition of Test Parameters

Percentage of SMS messages delivered within 15 seconds refers to SMS messages that were successfully sent from one network and received by another or within the same network in less than 15 seconds.

Percentage of SMS messages delivered within 15 minutes refers to SMS messages that were successfully sent from one network and received by another or within the same network in less than 15 minutes.

Percentage of SMS messages delivered within 12 hours refers to SMS messages that were successfully sent from one network and received by another or within the same network in less than 12 hours.

Methodology

Since 2009, the test comprises both the stationary and mobility test modes. Each test period is randomly selected by IMDA.

During the stationary mode test period, SMS messages are sent and received within and between all three mobile operators' networks for 24 hours a day from/at a fixed location, with an interval of one minute between each SMS message.

During the mobility mode test period, SMS messages are sent and received within and between all three mobile operators' networks, with an interval of one minute between each SMS message. This test is conducted outdoors in a moving vehicle on an islandwide basis.

The test uses the same handset model for all mobile operators to avoid possible distortion of the test results due to technical differences in handsets. Two handsets are used for each mobile operator, one for sending the SMS message and another for receiving the SMS message sent.

The test is conducted using a standard maximum message length of 160 characters*. All handsets are placed in a fixed location or in a vehicle for stationary and mobility tests, respectively.

All test messages are computer-generated and collection of results is fully automated.

At the end of each test period, timings are consolidated and analysed.

Interpretation of Results

In interpreting the results, it is noted that:

  • The SMS performance tests results represent a snapshot of the SMS performance during the time of testing. The reported results may not reflect end users' experience at all time.
  • The inter-operator SMS performance is dependent on the originating and terminating networks. Network failure or congestion in any of the two networks will affect the inter-operator SMS performance.

Survey Results

* According to the Global System for Mobile communication ("GSM") standards, a standard SMS message usually has a maximum length of 160 characters.

Last updated on: 18 Jul 2019