Yes, the Plugin approval team is more than right, this is not the way to do it.
First a couple of articles from a core developer and an excellent plugin developer, where you'll get to know why not to use wp-load.php
:
Don’t include wp-load, please, from Otto on WordPress
wp-load.php – I Will Find You!, from Pippin'g blog Crappy Code.
When initiating your plugin, you can define the plugin's URL as follows:
$plugin_url = plugins_url( '/', __FILE__ );
And when calling your download script, you can pass the needed information in a query var, e.g., $plugin_url . 'download.php?file=FILE-URL'
and $_GET
it in your script.
This is just a general outline, more details on your plugin's logic would need to be provided if this outline is not enough. And, yes, as @AJHenderson points out, WordPress Answers is the place where you'll find lots of developers, including Otto and Pippin.
Security notice
#Security notice YouYou need to have your download script tightly secured, as one could do the following:
http://example.com/path/to/plugin/download.php?file=http://example.com/wp-config.php