At least one of the back towers was added in later times. The Normans repaired and added to a heavily damaged Roman Tower which was built between 250AD and about 300AD, reflecting its late Roman design.
Look at this quote I just found in Gildas, regarding towers built in Southern England! I am unsure if it actually relates to the Roman Tower of London... but it would seem that preventing ships sailing up the Thames would resist a Barbarian attack!
"18... The Romans, therefore, left the country, giving notice that they could no longer be harassed by such laborious expeditions, nor suffer the Roman standards, with so large and brave an army, to be worn out by sea and land by fighting against these un-warlike, plundering vagabonds; but that the islanders, inuring themselves to warlike weapons, and bravely fighting, should valiantly protect their country, their property, wives and children, and, what is dearer than these, their liberty and lives; that they should not suffer their hands to be tied behind their backs by a nation which, unless they were enervated by idleness and sloth, was not more powerful than themselves, but that they should arm those hands with buckler, sword, and spear, ready for the field of battle; and, because they thought this also of advantage to the people they were about to leave, they, with the help of the miserable natives, built a wall different from the former, by public and private contributions, and of the same structure as walls generally, extending in a straight line from sea to sea, between some cities, which, from fear of their enemies, had there by chance been built. They then give energetic counsel to the timorous natives, and leave them patterns by which to manufacture arms. Moreover, on the south coast where their vessels lay, as there was some apprehension lest the barbarians might land, they erected towers at stated intervals, commanding a prospect of the sea; and then left the island never to return."
This sounds a lot like Rome in the time of Aetius, in the 450s. Rome had lost Africa to the Vandals a generation or so earlier and was struggling to maintain the remainder against a horde of barbarians with a much reduced income (only 40,000 pounds of gold per annum), which could only maintain an army at maximum of 30,000 soldiers, far too small. Even so 2/3 of its annual income was required for this!
By this stage, the Roman emperor only wished to reside in Ravenna, not liking the squalor of Rome. I believe the tower possibly went up in the late 200s or the 300s, at about the same time the walls around Chichester. Any later and the Romano-British would have had to have built it for themselves and I am unsure if the economy would have been powerful enough for such an endeavour.
In addition there was a building there called the Arx Palatina. (Palatine Fortress). There was another Arx Palatina on the other side of Londonium on the river bank, where Blackfriars now is. I believe this was converted into Blackfriars Monastery but I would have to look into this further.
New MAIN SITE: http://www.charleskos.com/
Old School Website http://charles.kos.id.au/
Videos collected together http://www.whatisgiza.com/
My publishing website: http://plusultrabooks.com.au/