Add caption functionality to custom wordpress theme

Hi I am working from a custom theme created by my predecessor and I am trying to add caption functionality to the theme but I am have some trouble with it.
Presently if I insert a image into a post and wish to add a caption, the caption would just appear next to the picture without any tags or classes containing it, here is a pic. I tried following another post and adding this piece of code to the function.php and to my single.php but nothing changed.

function the_post_thumbnail_caption() {
  global $post;

  $thumbnail_id    = get_post_thumbnail_id($post->ID);
  $thumbnail_image = get_posts(array('p' => $thumbnail_id, 'post_type' => 'attachment'));

  if ($thumbnail_image && isset($thumbnail_image[0])) {
    echo '<span>'.$thumbnail_image[0]->post_excerpt.'</span>';
  }
}

<?php the_post_thumbnail(); ?>

What do I have to include to implement captions properly under my images?