Stop wordpress to redirecting home page if no page found

Question

I don't want to redirect to home page when there is no page or wrong url. I have following in httaccess file

# BEGIN WordPress
RewriteRule ^sitemap.xml sitemap-%{SERVER_NAME}.xml [L]
RewriteRule ^sitemap.xml.gz sitemap-%{SERVER_NAME}.xml.gz [L]
RewriteRule ^index\.php$ - [L]
ErrorDocument 404 /404.php
ErrorDocument 404 /index.php?error=404

# uploaded files
RewriteRule ^([_0-9a-zA-Z-]+/)?files/(.+) wp-includes/ms-files.php?file=$2 [L]

# add a trailing slash to /wp-admin
RewriteRule ^([_0-9a-zA-Z-]+/)?wp-admin$ $1wp-admin/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ - [L]
RewriteRule  ^([_0-9a-zA-Z-]+/)?(wp-(content|admin|includes).*) $2 [L]
RewriteRule  ^([_0-9a-zA-Z-]+/)?(.*\.php)$ $2 [L]
RewriteRule . index.php [L]


# END WordPress

I have already 404.php in my theme folder. Let me know what to modify.