This is normal behavior. On Windows devices, you'll experience a +/- 30 second delay between what the presenter is doing and what the attendees see. On Mac and iOS devices, the delay will be around 60 seconds. this delay is due to the time it takes to convert to various codecs and distribute the media to Points of Presence servers within the Aure Content Delivery Network (CDN).
Curious why the delay makes it "unusable". Anyone who is part of the production team and using the Skype client should have near-zero latency, just like a regular Skype meeting. The audience shouldn't be able to tell that there's a delay (unless there's a clock in the view or they can see the broadcasters).
As @Erwin Bierens mentioned, once the broadcast is sent up to Microsoft, they have to re-encode the video to different formats and bitrates, apply any transcription and captioning you've selected, and then send that out to the CDN. Each of these steps takes processing time and any other live streaming service capable of handling viewers of this scale will have also have processing delays, so it's not something unique to Skype Broadcast.
This is comes from a lack of understanding of the product and use cases on my part. Our goal was to have a panel speaking to a larger audience. Some of the audience was going to be in attendance physically while others were online. Clearly this mixed group is not the appropriate model for broadcast. We had about 45 people and wanted to use multiple cameras to go from panel member to panel member, utilize the questions feature and have the content/video layout. It was unusable as the in house participants were seeing things in the "production room" time and couldn't handle the delay. Clearly this wasn't a good way to use the product. The facilitation of medium sized interactive meetings is still a bit of a challenge.