The Roman influence is most obvious in the well preserved amphitheatre.
The Venetian star fortress dominates the city and incorporates a small museum.
The best example of Austro-Hungarian fortifications is Fort Bourguignon. This is south of the city and tricky to find. Go through the reception barrier at Hotel Splendid and turn sharp right into their car park. You will see a pyramid structure on top of the hill and the fort is next to it. The guide book says that there are galleries with details of the fortifications around the city, but it was closed. I suspect the lack of signs means this is permanent.
If its medieval castles that interest you, there are several north of the city around Pazin. This is Pazin's medieval castle, the setting for a Jules Verne novel.
A ruin at Sumber.
and finally Krsan Castle.