I have created one post and post ID is 34 and added some Tags and Now I want to retrieve all tags with post ID.
Put this inside your loop:
Slightly adapted from http://codex.wordpress.org/Function_Reference/wp_get_post_terms#Examples
This is how you can retrieve tags
Customize it the way you like.
You can put this code within or outsode the default loop.