You may run out of RAM. The trick is to build less files in parallel. Like the example on "Build from source" section
here, set
export MAKEFLAGS="-j1"
This will create only one compilation process per package. You can also limit the number of packages built in parallel with:
colcon build --executor sequential
or
colcon build --parallel-workers 1