In Explicit Proxy deployment mode:
Proxy will respond with a http-407 proxy authentication requirement to client.
Internet Explorer (IE) automatically sends Windows credentials in the Proxy-Authorization: header when the ProxySG issues a challenge for NTLM/IWA.
In Transparent Proxy deployment mode:
Proxy will respond with a http-401 web authentication requirement to client.
IE does not offer Windows credentials in the Proxy-Authorization: header when the Proxy issues a challenge for NTLM/IWA unless the browser is configured to do so. In this case, the behavior is the same as for explicit proxy.
If IE is not configured to offer Windows credentials, the browser prompts for username/password, allowing non-domain users to be authenticated as guests in the policy substitution realm by entering worthless credentials.
So you will always get an authentication pop up, unless you configure your browser to offer Windows credentials to this http-401 response.
For Firefox, you can try to change the value of network.auth.force-generic-ntlm to true.
For IE, you can try to change the User Authentication value to “Automatic logon with current user name and password” in Internet Options->Security->Internet->Custom level.