It is possible to use ssh agent forwarding capabilities with scp.
Unfortunately, there is no short argument for scp like ssh -A, so you can use generic -o (option) argument like this:
$ scp -o "ForwardAgent yes" /path/to/file user@server:/path/to/dest