The_Raven
Posts: 27
Joined: Mon Sep 03, 2018 9:56 am
Location: Switzerland

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 8:21 am

We can talk now for a long time about it. But there are 2 things:
1. The PoE Hat ist faulty. Measuring shows this. And if you don't have USB-Devices, yes, it works, but what if you like to connect USB-Devices later? And what if you realize then that there is a problem? Yes, then your HAT is out of warranty. :roll:
2. It is not easy for any manufacturer to say "we made a fault". It is difficult for me to find the right words, english is not my native language. I find it great that the devs here say that it is faulty. This makes for me the difference between a good and a bad company. But i find it should told by a blog post.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 8:52 am

Yes, it doesn't work correctly in some circumstance. Finding those exact circumstances (we could not replicate the issue in house!) took a while with some great help from community members. Just as an anecdote, some community members sent guaranteed non-working devices, including the Pi3 they were attached too, to us for testing. We could not make them go wrong. Any of them, from multiple sources. That how odd this error is. Let me repeat that, kit that showed the error every single time they were turned on in the users home, when we tried them in the office, did NOT show the error. Exactly the same Pi3+ and PoE HAT, same SD card etc. Weird.

We do though, have a fix, which has been tested by the same community people with problems, and shown to work, and until that fix is applied, sales are suspended. It does take time to make the modification and refill the channels, hopefully just a few weeks. Sorry about the time it takes, but we need to get things right, and the process takes time.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

the_ewan
Posts: 14
Joined: Sat Oct 06, 2018 9:00 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:16 am

Sure, I think everyone gets that. That's all totally fine.

The problem is that there isn't a blog post (and tweets etc.) simply saying what you've said here - there's a problem, sales are suspended, there will be a fix, it'll be a few weeks.

There's no disagreement about the actual issue, just whether the Foundation is being obvious enough about letting people know.

The only post on the blog about the PoE hat is "INTRODUCING THE POE HAT – AVAILABLE NOW!" and, well, it's just not.

A one line correction on that post would be useful. Right now it's easy enough to find the details if you already know there's a problem. But if you don't know there's a problem at all it's easy to miss it completely.

sanhall
Posts: 1
Joined: Fri Oct 19, 2018 8:54 am

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:19 am

I don't think there's been any problem with the communication between engineers and community in this thread, in fact it's been fantastic! I can appreciate how tricky this must be to work out, I have one compatible and one non-compatible Pi that were bought at the same time. For the record this issue also occurs with the pi camera, not just the physical USB ports, I'm sure you already guessed that though.

The lack of an official blog post is terrible, it just needs to state that the first batch had some compatibility issues and is being held back - we don't need a full replacement timeline immediately. "Out of Stock" is a lie, "Coming Soon" is a misleading half-truth. A 9+ page forum thread where the technical issues were debugged is not a source of truth for the current plan. Who knows which of the old posts during the investigation phase are still true, you'd expect there to be different decisions being made over time.

Without an official announcement to wrap it up the official forum staff are just being thrown under the bus - if the media picks up on it who else is there to quote? It's been a whole month since the hold on sales was mentioned in this thread, an official summary can't be seen as an unreasonable expectation.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:23 am

the_ewan wrote:
Fri Oct 19, 2018 9:16 am
Sure, I think everyone gets that. That's all totally fine.

The problem is that there isn't a blog post (and tweets etc.) simply saying what you've said here - there's a problem, sales are suspended, there will be a fix, it'll be a few weeks.

There's no disagreement about the actual issue, just whether the Foundation is being obvious enough about letting people know.

The only post on the blog about the PoE hat is "INTRODUCING THE POE HAT – AVAILABLE NOW!" and, well, it's just not.

A one line correction on that post would be useful. Right now it's easy enough to find the details if you already know there's a problem. But if you don't know there's a problem at all it's easy to miss it completely.
I would expect any one doing a search will come to this thread just as easily whether the data were on the blog or not. Add a new blog post and it would fall off the front in a day, and never be seen again. A Google search for "raspberry Pi PoE HAT overcurrent" shows the first hit is this thread.

I'll see if the whether the original blog post can be updated to point to this thread though.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

The_Raven
Posts: 27
Joined: Mon Sep 03, 2018 9:56 am
Location: Switzerland

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:24 am

Interesting. But i think if it works or not is not so important, important is the measuring. It is not a clean DC output, it has a lot of ripples. Devs sould have a focus on this and trya to remove the ripples and make a clean DC output.
Only with a clean DC output runs the pi stable. ;)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:25 am

The_Raven wrote:
Fri Oct 19, 2018 9:24 am
Interesting. But i think if it works or not is not so important, important is the measuring. It is not a clean DC output, it has a lot of ripples. Devs sould have a focus on this and trya to remove the ripples and make a clean DC output.
Only with a clean DC output runs the pi stable. ;)
We have fixed the issue.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

the_ewan
Posts: 14
Joined: Sat Oct 06, 2018 9:00 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:36 am

A Google search for "raspberry Pi PoE HAT overcurrent"
Oh, come on - there's no way anyone can run that search until they not only know there's a problem, but something about the specific nature of the problem.

Someone who just wants a hat and has no idea why they're out of stock most places isn't going to run that search.

Google just "raspberry pi PoE hat" instead and the top hit is the Foundation's product page, and the second is the blog post which says 'available now', and neither of which mention the problem. This thread doesn't feature on the first page of results.

That's why those pages should have an update noting the issue.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 9:44 am

the_ewan wrote:
Fri Oct 19, 2018 9:36 am
A Google search for "raspberry Pi PoE HAT overcurrent"
Oh, come on - there's no way anyone can run that search until they not only know there's a problem, but something about the specific nature of the problem.

Someone who just wants a hat and has no idea why they're out of stock most places isn't going to run that search.

Google just "raspberry pi PoE hat" instead and the top hit is the Foundation's product page, and the second is the blog post which says 'available now', and neither of which mention the problem. This thread doesn't feature on the first page of results.

That's why those pages should have an update noting the issue.
Replace "overcurrent" (which is the warning in the dmesg log) with "problem" and you still get a bunch of valid hits, with this thread being the third in the list. Replace with things like "not available" or "not working" get similar lists of hits. I don't think having a blog post would make the issue any more visible, but I have asked (the big boss), and it hasn't happened.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Mortimer
Posts: 924
Joined: Sun Jun 10, 2012 3:57 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 10:46 am

Surely it's up to the sales outlets how they present information about a product and what they say. If their online sales engine and website doesn't have any other appropriate item status than "out-of-stock", then what are they to do? It is technically correct.
--------------
The purpose of a little toe is to ensure you keep your furniture in the right place.

DjoeC
Posts: 9
Joined: Tue Dec 23, 2014 8:47 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 11:52 am

I LIKE to be an early adopter but did not go for a pre-order or even an immediate order when available. Reason: Time between announcement and availability was too long - to me a warning sign. Then when they became available I waited a few weeks to first check for Raspberry POE hat Problems in a search engine. Luckily there were some postings therefore I haven't bought (yet).

The consequences are that my (outdoor mounting) project has been on hold for a long time.

The good thing is that support and openness in this thread is good and we are being informed. Thumbs up!
The bad thing is that version 1 (or is it 0.99) of the POE hat is still on sale without warning for the less wary people.

I hope to see a physically identifiable improved version coming soon! Good luck guys and galls!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 12:22 pm

DjoeC wrote:
Fri Oct 19, 2018 11:52 am
I LIKE to be an early adopter but did not go for a pre-order or even an immediate order when available. Reason: Time between announcement and availability was too long - to me a warning sign. Then when they became available I waited a few weeks to first check for Raspberry POE hat Problems in a search engine. Luckily there were some postings therefore I haven't bought (yet).

The consequences are that my (outdoor mounting) project has been on hold for a long time.

The good thing is that support and openness in this thread is good and we are being informed. Thumbs up!
The bad thing is that version 1 (or is it 0.99) of the POE hat is still on sale without warning for the less wary people.

I hope to see a physically identifiable improved version coming soon! Good luck guys and galls!
Time between announcement and launch is almost always just to ramp up production and fill up the channel. If you are familiar with the process of getting a production line (with associated test stations) up and running you will know how long this can take. This is one of the major reasons why almost every HW kickstarter ever has taken much longer to get to market that originally planned. The people doing them massively underestimate the timescales required, because on the whole they have no experience in it.

That said, we generally don't announce product in advance but in the PoE case, the announcement was forced by the appearance of the Pi3B+ and its PoE plug. Since we have limited staff, the PoE HAT ramp up had to happen after the Pi3B+ release (which of course was NOT announced in advance). Don't take the time taken as a warning sign, it's just the way it is. Ironically, had we spent more time on it, perhaps this mistake would have been detected prior to launch....
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Roken
Posts: 309
Joined: Sun Dec 31, 2017 4:35 pm
Location: UK

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 11:10 pm

the_ewan wrote:
Fri Oct 19, 2018 9:36 am
A Google search for "raspberry Pi PoE HAT overcurrent"
Oh, come on - there's no way anyone can run that search until they not only know there's a problem, but something about the specific nature of the problem.

Someone who just wants a hat and has no idea why they're out of stock most places isn't going to run that search.

Google just "raspberry pi PoE hat" instead and the top hit is the Foundation's product page, and the second is the blog post which says 'available now', and neither of which mention the problem. This thread doesn't feature on the first page of results.

That's why those pages should have an update noting the issue.
Then just try "pi poe hat problem", which I would generally do for a new product that I was planning on buying. It's my go to type search.
Headless PI. OMG, someone cut it's head off. Oh, hang on. it didn't have one to start with.

the_ewan
Posts: 14
Joined: Sat Oct 06, 2018 9:00 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 11:16 pm

Or possibly the Foundation could just make it as easy as possible, rather than sticking to the 'bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying Beware of the Leopard' approach and expecting everyone else to work it out.

This obviously isn't a secret, there's clearly no cover-up, so what's the harm in making it a bit easier and simpler?

martinrowan
Posts: 71
Joined: Wed Aug 26, 2015 11:00 am
Location: Berkshire, UK
Contact: Website Twitter

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 19, 2018 11:36 pm

jamesh wrote:
Fri Oct 19, 2018 8:52 am
Yes, it doesn't work correctly in some circumstance. Finding those exact circumstances (we could not replicate the issue in house!) took a while with some great help from community members. Just as an anecdote, some community members sent guaranteed non-working devices, including the Pi3 they were attached too, to us for testing. We could not make them go wrong. Any of them, from multiple sources. That how odd this error is. Let me repeat that, kit that showed the error every single time they were turned on in the users home, when we tried them in the office, did NOT show the error. Exactly the same Pi3+ and PoE HAT, same SD card etc. Weird.
James, you or any team fathers team are welcome to pop round for a cuppa, inn order to test and measure any of the faulty pair of Pi+HAT, in my home network. Or you could send back a modified HAT on the combo that showed the issue to confirm that the proposed fix address all situations. Better to do that, than need to release a v3 version, if we managed to turn up something else. I know my home network has mains 240V cables running closer to the network cables than some other installations, but this shouldn't cause an issue. I assume the testing by the foundation team isn't just relying on lab PSU, or ideal network conditions and instead is experimenting with the real life variances that this type of hardware would be expected to handle.

Cheers Martin

pantner
Posts: 13
Joined: Mon Oct 08, 2012 5:47 am

Re: PoE HAT - USB Ports not working - over-current

Mon Oct 22, 2018 6:16 am

Ok, so, I'm a little annoyed as well.

I was able to order this product on the 19th Sept, I had not thought to do any google searches to find out if there are any problems.

In between ordering (and shipping) of the HAT and it arriving i mentioned it on a forum thread i am a part of, a few people there made mention of power issues which then lead me here.

Since the only USB Device i use is a reciever for a wireless keyboard, i was hoping i would have good luck and it would work.

Unfortunately that is not the case. I was able to boot up the Pi (running the latest image of OSMC) and do the basic setup.
In the beginning it appeared to work, however when trying to play a specific video file, I had what I believe to be a kernel panic.
The same file then played ok after the Pi rebooted.

Went to bed, woke up in the morning, Pi was just frozen. I power cycled the Pi, within 30 seconds it had frozen again (while I was tying to browse my media library).
I switched back to a non-POE switch, plugged my USB power back in and it has been running flawlessly. (Apple iPad charger, not the best but it works)

The Pi, SD Card and HAT were all new out of the box (before this i had a Pi 2 running).

Is there anything that can be done?

User avatar
Burngate
Posts: 6090
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: PoE HAT - USB Ports not working - over-current

Mon Oct 22, 2018 8:32 am

pantner wrote:
Mon Oct 22, 2018 6:16 am
Is there anything that can be done?
Other than send it back for replacement, probably not.
It may depend on how much information we get about the gory details of what the problem turned out to be.

I'm intrigued by James's anectdote:
... some community members sent guaranteed non-working devices, including the Pi3 they were attached too, to us for testing. We could not make them go wrong.
Also, the report from someone managing to stop the problem by stalling the fan with his finger.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Mon Oct 22, 2018 8:55 am

Burngate wrote:
Mon Oct 22, 2018 8:32 am
pantner wrote:
Mon Oct 22, 2018 6:16 am
Is there anything that can be done?
Other than send it back for replacement, probably not.
It may depend on how much information we get about the gory details of what the problem turned out to be.

I'm intrigued by James's anectdote:
... some community members sent guaranteed non-working devices, including the Pi3 they were attached too, to us for testing. We could not make them go wrong.
Also, the report from someone managing to stop the problem by stalling the fan with his finger.
Like I said, it's a weird one, but we do have a fix and are working on getting it out there asap.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

pantner
Posts: 13
Joined: Mon Oct 08, 2012 5:47 am

Re: PoE HAT - USB Ports not working - over-current

Mon Oct 22, 2018 9:09 am

jamesh wrote:
Mon Oct 22, 2018 8:55 am
Like I said, it's a weird one, but we do have a fix and are working on getting it out there asap.
Is there any point holding on to what I have now? Or should I just be returning it?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Mon Oct 22, 2018 9:10 am

pantner wrote:
Mon Oct 22, 2018 6:16 am
Ok, so, I'm a little annoyed as well.

I was able to order this product on the 19th Sept, I had not thought to do any google searches to find out if there are any problems.

In between ordering (and shipping) of the HAT and it arriving i mentioned it on a forum thread i am a part of, a few people there made mention of power issues which then lead me here.

Since the only USB Device i use is a reciever for a wireless keyboard, i was hoping i would have good luck and it would work.

Unfortunately that is not the case. I was able to boot up the Pi (running the latest image of OSMC) and do the basic setup.
In the beginning it appeared to work, however when trying to play a specific video file, I had what I believe to be a kernel panic.
The same file then played ok after the Pi rebooted.

Went to bed, woke up in the morning, Pi was just frozen. I power cycled the Pi, within 30 seconds it had frozen again (while I was tying to browse my media library).
I switched back to a non-POE switch, plugged my USB power back in and it has been running flawlessly. (Apple iPad charger, not the best but it works)

The Pi, SD Card and HAT were all new out of the box (before this i had a Pi 2 running).

Is there anything that can be done?
Our recomendation if you are seeing issues is to return for a refund. The new version should be out soon.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Paul Webster
Posts: 812
Joined: Sat Jul 30, 2011 4:49 am
Location: London, UK
Contact: Twitter

Re: PoE HAT - USB Ports not working - over-current

Tue Oct 23, 2018 4:35 pm

I saw one supplier announcing shipping (re-)starting yesterday.

Edit:
Correction ... the notification that Insaw was for a POE HAT starting to ship but it was not the one from RPT/RPF

DjoeC
Posts: 9
Joined: Tue Dec 23, 2014 8:47 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 26, 2018 2:19 pm

James,
Like I said, it's a weird one, but we do have a fix and are working on getting it out there asap.
Could you provide some extra insight? Is it a problem with the Poe HAT, the 3B+, the firmware? And: how to recognize the fixed items with expected time to market? I would think many resellers will try to offload the problematic parts.....

Will there be any communication about specific cause and solution forthcoming?

Thanks!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24129
Joined: Sat Jul 30, 2011 7:41 pm

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 26, 2018 2:42 pm

DjoeC wrote:
Fri Oct 26, 2018 2:19 pm
James,
Like I said, it's a weird one, but we do have a fix and are working on getting it out there asap.
Could you provide some extra insight? Is it a problem with the Poe HAT, the 3B+, the firmware? And: how to recognize the fixed items with expected time to market? I would think many resellers will try to offload the problematic parts.....

Will there be any communication about specific cause and solution forthcoming?

Thanks!
It's a hardware fault, requiring a fix to the HW. All board sent back to us by resellers will be fixed, so the majority of resellers will not be 'offloading problematic parts'. Future production runs will have a updated PCB anda version number bump. The fix is obvious*, so if anyone is selling on, then it will be easily spotted, although worth noting that for most people there is no issue.

Not sure of timescales, but must be pretty close.


* Although I just took a look at one, and had to have the fix pointed out. Once pointed out, its obvious!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
Burngate
Posts: 6090
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK Tralfamadore
Contact: Website

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 26, 2018 5:34 pm

jamesh wrote:
Fri Oct 26, 2018 2:42 pm
It's a hardware fault, ... All board sent back to us by resellers will be fixed ... Future production runs will have a updated PCB and a version number bump. The fix is obvious*, so if anyone is selling on, then it will be easily spotted.

* Although I just took a look at one, and had to have the fix pointed out. Once pointed out, its obvious!
So an "old" board can be fixed, but the "new" boards will have a different pcb, anyway?

I hope, when a blog comes along, we get a couple of photos to show us the fix. Possibly a "spot the difference" competition. With prizes, naturally. A cuddly toy for runners-up; what should the winner get?

User avatar
DougieLawson
Posts: 36526
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: PoE HAT - USB Ports not working - over-current

Fri Oct 26, 2018 5:48 pm

Burngate wrote:
Fri Oct 26, 2018 5:34 pm
So an "old" board can be fixed, but the "new" boards will have a different pcb, anyway?

I hope, when a blog comes along, we get a couple of photos to show us the fix. Possibly a "spot the difference" competition. With prizes, naturally. A cuddly toy for runners-up; what should the winner get?
The obvious prize would be the re-designed "VNext" PoE HAT.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

Return to “HATs and other add-ons”