In general, you will find the latest version on http://www.octave.org/download.html. It is recommended to use the “stable” version of octave for general use, and the “development” version if you want the latest features.
A list of user-visible changes since the last release is available in the file NEWS. The file ChangeLog in the source distribution contains a more detailed record of changes made since the last release.