There is a way to do it without hacking your theme - Settings -> Reading - > Blog pages show at most [X] Posts. The downside to this setting is that it affects archive pages and search pages too, which is mega suckage.

To just display the latest post on your homepage, change the "while" statement to an "if" statement. So, instead of:

<?php while (have_posts()) : the_post(); ?>
[ Post Stuff ]
<?php endwhile; ?>

...with:

<?php if (have_posts()) : the_post(); ?>
[ Post Stuff ]
<?php endif; ?>

I've changed the "while" on the first line to "if", and to close it i've changed the "endwhile" to "endif". How does this work? Well, the while statement continues to print posts on your hompage until it exhausts the post data it's given from WordPress with a loop, the if statement just prints the first post it's given from the data because it's not told to carry on; there is no loop.