When Part 103 was written it was assumed powered ultralights would use a gas engine so the limitation in the regulation is based upon fuel capacity in gallons. There was no, and currently is no consideration for electric powered ultralights. So, an electric powered ultralight must comply with the current Part 103 rules, for example:
254 pounds max empty weight
55 knots max full power level flight speed
24 knots max power off stall speed
Extra 24 pounds (278 pounds total) if ballistic chute is installed
5 gallons max fuel capacity (N/A to electric powered ultralight)
So, the question is what about battery weight? It is included in the empty weight of the ultralight no extra allowance is provided for.
It would seem reasonable that the FAA would consider an exemption to allow some extra weight for the weight of the batteries.
Ultralights seem like a great fit for the development of electric powered flying machines, good luck with your project!