The research on lightweight concrete has been an area of interest, especially in modular construction since low self-weight can ease the transportation process and speed up construction time. Past research found that conventional lightweight aggregate concrete (LWAC) developed using coarse lightweight aggregates and natural fine aggregates can fulfil the requirement of structural lightweight concrete. However, the dry densities achieved by these LWAC are relatively high, at around 1750–2070 kg/m3. To address this issue while producing structural LWAC with a lower density range, micro lightweight aggregates (MLWA) with closed pore structures are utilized. In this study, two types of MLWA: fly ash cenospheres (FAC) and perlite microspheres (PM) were used to produce two series of lightweight cementitious composite (LCC). The incorporation level of MLWA was 25%, 50%, 75% and 100% by volume of coarse silica sand which was utilized to optimize the content of MLWA. The performance of LCC was investigated through compressive strength, flexural strength, water absorption, drying shrinkage and thermal conductivity tests. Use of FAC and PM yielded satisfactory compressive strength of up to 46.7 MPa and 45.3 MPa respectively at low densities. While flexural strength of mixes with FAC and PM reached 8.0 MPa and 8.7 MPa respectively. Use of 75% FAC showed the lowest water absorption of 3.3%. Incorporation of 100% PM reduced thermal conductivity by as much as 82.4%. The optimum LCC mix in this study comprised 75% FAC and 25% coarse silica sand which achieved a specific strength of 29.0 kPa/kgm−3 and a dry density of 1406 kg/m3, while having an average compressive strength and flexural strength of 40.8 MPa and 6.3 MPa at 28 days, respectively. The overall results show that LCC made up of FAC is more suitable for structural applications, whereas LCC made up of PM exhibits better thermal insulation properties.