So you're saying that in DHCP you have 10.4.20.1/19 range setup but you are only handing out IP addresses in 10.4.20.1 - 10.4.21.250?
You could recreated your DHCP scope to only list want you actually want to use as DHCP addresses. Another option is to turn on 'Enable Duplicated Subnets' so that you can add them in manually as well into IPAM.
Please let us know how you sorted your issue.