Note that Coinbase doesn't necessarily create a transaction immediately with your withdrawal. So some of the delay could be waiting for the bank to actually broadcast the withdrawal.