There's a very good reason why you should add all your wallets and transactions.
Let's look at an example, say you bought 10 BTC on Coinbase for $500 and sent them to your Ledger wallet and a few days after that, you transferred these BTC from Ledger to Binance and sold them for $2000. Now, if you only added your Coinbase and Binance wallets but forgot to add Ledger then your transactions will look like this:
As you can see Koinly thinks the transfer from Coinbase to Ledger is a withdrawal or sale as it can't possibly know you transferred it to another one of your own wallets! So, it uses the market price on the date of the Withdrawal to determine the profit/loss.
Next, the deposit of 10 BTC into Binance has got a value of $3000 which was the market price for the BTC on the day of the deposit.
Finally, when you sell the 10 BTC, Koinly assumes you had initially bought the coins for $3000 and calculates a loss...
Now let's add the Ledger wallet into the mix, here's how the transactions look now:
Well, that looks better, doesn't it?
Koinly has matched the transfer from Coinbase to Ledger and also the transfer from Ledger to Binance, so Koinly knows that you bought your BTC for $500 on Coinbase even though you are selling it on Binance!
This is why you really need to add all of your transactions from all the wallets you traded on and ensure transfers between them are matched - Koinly can match most of your wallet transactions automatically as long as the transfer-matching criterias are met.