To whom is Ulysses speaking in "Ulysses"?

The speaker, the fictional Ulysses, is speaking to an
implied audience in Tennyson's poem named after him.   The poem is a dramatic monologue,
although the silent listener is an implied audience--so there is more than one silent
listener.

Ulysses says "You and I are old."  The "you,"
here, is presumably plural.  He may be addressing his former followers who returned home
with him after his adventures, although this would alter the story as written by Homer
in The Odyssey." 

Ulysses also
addresses his audience as "my friends."  Thus, the reader knows he is addressing more
than one person.  The poetic dramatic monologue form was new at the time Tennyson wrote
the poem, so the convention of only addressing one silent listener was not well
established. 

In short, then, Ulysses is talking a group of
followers, old men as he is, into going on an adventure with
him.