Limit upload by file type only for certain custom post type

Question

I have created a custom post type called "pdf". I've have a custom metabox where user can upload a pdf file throught wordpress builtin media uploader I need to restrict allowed files (pdf) only in this uploader. I already know how to restrict allowed files: How to set file type in wp_handle_upload? but the link above, interfer with all the uploads. Any hint?

Thanks

Edit: I've found how to limit file type in plupload, so I've solved half of the problem:

 add_filter( 'plupload_init', 'my_plupload_init', 0, 1 );

 function my_plupload_init( $plupload_init ) {
 $plupload_init['filters'] = array( array('title' => __( 'Allowed Files' ), 'extensions' => 'pdf') );
 return $plupload_init;
}

Edit #2: Use of wp_handle_upload_prefilter filter doesn't work: Thi is what is available inside $_REQUEST object after an image is uploaded on a new post:

    array
  'type' => string 'image' (length=5)
  'tab' => string 'type' (length=4)
  'post_id' => string '0' (length=1)
  '_wpnonce' => string '375c5f46f2' (length=10)
  '_wp_http_referer' => string '/wp-admin/media-upload.php?post_id=' (length=37)
  'html-upload' => string 'Carica media' (length=12)

There are no info about my custom post type. The id is 0 because post is not saved yet.

Edit #3:

Thanks to @TheDeadMusic answer I've added one line of code to my javascript. This is the js code I actually use that works great:

jQuery(document).ready(function($){
// #btn_upload is my custom button
$('#btn_upload').click(function(event){
    event.preventDefault();
    var backup = window.send_to_editor;
    var src=''; 
//post_id is always set so I can pass it safely to tickbox
    var post_id=$('#post_ID').val(); 
var target=$('#pdf_url');
    window.send_to_editor = function(html) 
     {
//I have to get the link to "original file" from html returned by tickbox
        link = $('a',html).attr('href');
         $(target).val(link);
         tb_remove();
         window.send_to_editor=backup; //reset default action
    }
    //show the uploader
     tb_show('', 'media-upload.php?post_id='+post_id+'&TB_iframe=true');

}); 

});

Answer 1

function wpse_59621_mimes_filter( $mimes ) {
    return array( 'pdf' => 'application/pdf' );
}

function wpse_59621_delay_mimes_filter( $value ) {
    if ( isset( $_REQUEST['post_id'] ) && get_post_type( $_REQUEST['post_id'] ) === 'my_post_type' )
        add_filter( 'upload_mimes', 'wpse_59621_mimes_filter' );
    else
        remove_filter( 'upload_mimes', 'wpse_59621_mimes_filter' );

    return $value;
}

add_filter( 'wp_handle_upload_prefilter', 'wpse_59621_delay_mimes_filter' );

Let us know how it goes - this is untested, but I'm confident!