If you are eligible for Medicare, you should not rely solely on VA benefits for your health care needs. Medicare does not work with VA benefits and if you receive care outside of a VA facility you might need Medicare. Medicare will not pay for any care received at a VA facility.
You should enroll in Medicare Part A and Part B to guarantee coverage outside the VA system. If you drop Medicare, you will have to get all your health care at VA facilities. Since the VA does not guarantee benefits to all veterans, you could be left with no coverage at all and you may not always be near a VA facility.