Blog

The Inter-university Institute for Data Intensive Astronomy is a partnership of three South African universities, the Universities of Cape Town, of the Western Cape and of Pretoria. IDIA also has an industry partner in SAP. The overarching goal of IDIA is to build within the South African university research community the capacity and expertise in data intensive research to enable global leadership on MeerKAT large survey science projects and large projects on other SKA pathfinder telescopes, leading to leadership on SKA phase 1 Key Science programs.

One of the first elements to reach this goal was for IDIA to set up a data-intensive research cloud facility to service its scientific community. Currently, IDIA is the primary platform to service five out of eight MeerKAT large survey projects.

Big scientific data changes the way we do science. Scientists themselves have to change their habits and approaches when data have to live in the cloud. Platforms like IDIA’s research cloud also need to enable scientists to keep applying the scientific method; visualising their data, re-processing it, testing hypotheses on it, etc. without having to wait for weeks for results because the data sets are so big.

Cloud technology also opens the door to real-time collaboration across the globe as analysis tools live in the cloud and can be run by collaborators on the same platform. Scientists also need assistance from technologies such as machine learning and artificial intelligence as the data are way too vast to look at manually.

The Inter-University Institute for Data Intensive Astronomy is here to answer some of those questions and facilitate research in the age of astronomical quantities of scientific data.

SCIENCE
IDIA was set up to offer astronomers in South Africa a platform on which to carry out research and training activities in the age of SKA. In July 2018, the inauguration of the MeerKAT telescope took place and IDIA researchers are already obtaining scientific discoveries.

The history of radio astronomy in South Africa is a recent but trailblazing one. After a successful bid to host the majority of the Square Kilometre Array telescope in 2012, the community has grown substantially, with many young South African scientists being trained by South African and foreign radio astronomers attracted to South African universities by the excellence demonstrated during the SKA bidding process.

MeerKAT is the South African Radio Astronomy Observatory (SARAO) SKA precursor telescope. It is operating as an independent radio telescope before being integrated into the first phase of the global SKA telescope. The South African research community is making excellent use of those years of the MeerKAT telescope; one of the best radio telescopes in the world is located right here.

In 2016, a workshop was organised by a collaboration of IDIA, SARAO, Rhodes University in the Eastern Cape and the University of KwaZulu-Natal to focus on the science that could be done with MeerKAT. At that time, eight MeerKAT Large Survey Projects (MLSPs) had already been nominated by SARAO for two thirds of MeerKAT’s first five years of operations. The MLSPs are ambitious scientific projects that require many hours of MeerKAT observations and open the door to entirely new scientific results.

In 2017, five of the MLSPs requested to use IDIA as their main data processing platform. In 2018, the first call for observing proposals for the remaining third of MeerKAT’s first five years of operations was issued by SARAO. Many researchers at IDIA partner universities were successful with their science proposals, and thus IDIA expects to become the platform of choice for most MeerKAT observations by those researchers and their collaborators.

TECHNOLOGY
One of the main goals of IDIA was to set up a technology platform to enable researchers in South Africa to do science with astronomical big data. Cloud technology opens the door to real-time collaboration across the globe as tools for analyses reside in the cloud and can be run by collaborators using the same platform though web access anywhere in the world. Big scientific data changes the way we do science.

Platforms like IDIA’s research cloud enable scientists to proceed with their scientific research; visualising their data, re-processing it, testing hypotheses, etc. without having to wait for weeks for results because the data sets are so big.

Scientists also need assistance from technologies such as machine learning and artificial intelligence as the data sets coming out of the new generation of telescopes are too vast to look at manually. Development work on the IDIA research cloud enables machine learning techniques to be embedded in data processing pipelines.

University of Western Cape UWC Inter University Institute for Data Intensive Astronomy IDIA Contact