You can just replace the html with a checkbox, ensuring the name of the checkbox is the name of the
<input> we're replacing.
You can also change the 'label' and the 'help' text, (see here for there the filter is fired) - just simply uncomment the appropritate lines and replace the text with your own.
The html we are replacing is normally provided by
image_link_input_fields (see here). Looking into that code we see the name of the input is
attachments[$post->ID][url] - which is the name we give our checkbox, with value the url of the file.
add_filter('attachment_fields_to_edit', 'my_attachment_fields_edit', 10, 2);
$file = wp_get_attachment_url($post->ID);//Get the file link
$url_type = get_user_setting('urlbutton', 'none');//Get user setting
$checked = checked($url_type,'file',false);//Automatically check box if setting is 'file'.
$html = "<input type='checkbox' name='attachments[".$post->ID."][url]' value='".esc_attr($file )."' ".$checked."/>";
$form_fields['url']['html'] = $html; //Replace html
//$form_fields['url']['label'] = __('Link URL');
//$form_fields['url']['helps'] = __('Enter a link URL or click above for presets.');
The above works perfectly, however, you may have noticed when selecting 'file url' or 'post url', or 'none' that WordPress remembers your choice and uses this as the default for next time. Your choice is stored in the user setting 'urlbutton'.
Unfortunately this method means that the setting is no longer updated (i.e. you can manually set the setting to 'file' or 'none', but it won't behave dynamically as before). I think the reason is that clicking the buttons triggers an AJAX request to update the settings.
$checked are a bit useless and can be removed.