Indeed, putin, the rhinoceros is one of several possible translations. (Even in English, these were occasionally referred to as unicorns back in the day).
That said, there's
(a) nothing to say one could not refer to the horn of a wild ox as "exalted" (certainly it's above his head, and it's such a poetic term in the first place that it's pretty hard to know what would or wouldn't be considered exalted), and
(b) if there are two of something, there is also one. Consider for example Psalm 94:9:
"Does he who fashioned the ear not hear?
Does he who formed the eye not see?"
Are we to infer that the Psalmist believed humans (and in particular, he himself) had only a single ear or eye? I would suggest not.
In fact, in Deut. 33, plural IS used of the horns of this same creature:
"A firstborn bull[k]—he has majesty,
and his horns are the horns of a [wild ox];
with them he shall gore the peoples,
all of them, to the ends of the earth;"
(I've bracketed the word in question).
Most telling is the fact that in several of the uses of this word, it's clear that imagery is being used that is supposed to appeal to the familiar in the readers (as you can tell, for example, from the context and the other comparisons being made).