Put your code in the template file `category.php`.

Remove all the part before the loop: once in category template, you don't need to get the category, get the paged, run again the query with `query_posts`...

So your `category.php` should *simply* appear like so:

    if ( have_posts() ) : while ( have_posts() ) : the_post(); 
      $image = wp_get_attachment_image_src( get_post_thumbnail_id( $post->ID ), 'single-post-thumbnail' );
    ?>
    
    ... post markup here
    
    <?php endwhile; 
    
    else: ?>
    
    <p><?php _e('No posts found'); ?></p>
    
    <?php endif; ?>
    
    <!-- Pagination Part -->
    <div id="pagination">
      <div class="next"><?php next_posts_link('next &raquo;') ?></div>
      <div class="prev"><?php previous_posts_link('&laquo; previous') ?></div>
    </div>

To force that template to show only 4 fosts per page, in `functions.php` use:

    add_action('pre_get_posts','four_post_per_cat');
    
    function four_post_per_cat( $query ) {
      if ( ! is_admin() && is_main_query() && is_category() ) {
        $query->set('posts_per_page', 4);
      }
    }

After that `numberposts` and `posts_per_page` are synonyms, but `numberposts` is deprecated. Setting different values for them make `numberposts` do nothing (or `posts_per_page` do nothing, I can't remember... however, use one of them).

If your scope is limiting the total posts reached (in all the pages), use the [`post_limit`][1] filter, in `functions.php` add also:

    add_filter( 'post_limits', 'cat_post_limits' );
    
    function cat_post_limits( limit ) {
        return ( is_category() ) ? 'LIMIT 0, 50' : $limit;
    }

Following my tips, not also you'll solve your issue, but also improve performace: because `query_posts` is very bad regarding performance: never use it.


----------


*PS: If you get at max 50 posts, 50 is not divisible by 4, so last page will have 2 posts.. why don't set limit to 52 or 48?*


  [1]: http://codex.wordpress.org/Plugin_API/Filter_Reference/post_limits