Lessons Learned From A CubeSat Postmortem

On the 3rd of June 2019, a 1U CubeSat developed by students of the AGH University of Science and Technology in Kraków was released from the International Space Station. Within a few hours it was clear something was wrong, and by July 30th, the satellite was barely functional. A number of problems contributed to the gradual degradation of the KRAKsat spacecraft, which the team has thoroughly documented in a recently released paper.

We all know, at least in a general sense, that building and operating a spacecraft is an exceptionally difficult task on a technical level. But reading through the 20-pages of “KRAKsat Lessons Learned” gives you practical examples of just how many things can go wrong.

KRAKsat being released from the ISS

It all started with a steadily decreasing battery voltage. The voltage was dropping slowly enough that the team knew the solar panels were doing something, but unfortunately the KRAKsat didn’t have a way of reporting their output. This made it difficult to diagnose the energy deficit, but the team believes the issue may have been that the tumbling of the spacecraft meant the panels weren’t exposed to the amount of direct sunlight they had anticipated.

This slow energy drain continued until the voltage dropped to the point that the power supply shut down, and that’s were things really started going south. Once the satellite shut down the batteries were able to start charging back up, which normally would have been a good thing. But unfortunately the KRAKsat had no mechanism to remain powered down once the voltage climbed back above the shutoff threshold. This caused the satellite to enter into and loop where it would reboot itself as many as 150 times per orbit (approximately 90 minutes).

The paper then goes into a laundry list of other problems that contributed to KRAKsat’s failure. For example, the satellite had redundant radios onboard, but the software on them wasn’t identical. When they needed to switch over to the secondary radio, they found that a glitch in its software meant it was unable to access some portions of the onboard flash storage. The team also identified the lack of a filesystem on the flash storage as another stumbling block; having to pull things out using a pointer and the specific memory address was a cumbersome and time consuming task made all the more difficult by the spacecraft’s deteriorating condition.

Of course, building a satellite that was able to operate for a couple weeks is still an impressive achievement for a student team. As we’ve seen recently, even the pros can run into some serious technical issues once the spacecraft leaves the lab and is operating on its own.

[Thanks to ppkt for the tip.]

35 thoughts on “Lessons Learned From A CubeSat Postmortem

  1. I would say that the satellite was a huge success. 1) it worked. 2) it failed in a way they understood 3) they were able to learn from it. I can’t imagine it going better than that. Why? Because “perfect success” doesn’t teach anything. What if it was just pure luck that nothing went wrong? What if it was only successful because someone was coaching them every step of the way- Where will that coach be in the future? Instead, the experience and lessons will be with the students forever, and it’ll affect how they approach things.

    1. It didn’t accomplish it’s scientific goals because the data couldn’t be retrieved from the sensors/payload. So it was definitely a failure in terms of the mission.

      The best we could say is that it functioned in a semi-useful state for a few days. If that’s a ” huge success”, we aren’t setting the bar very high there.

      1. exactly
        such spaceship should be pretested on the ground in simulated space conditions for 365 day to save on launching costs (standard procedure)

        If project goes wrong, so money and time are lost.

    2. But they don’t really know why it failed. They have a list of things that went wrong, but they lacked the sensors to determine what actually prevented the batteries from charging in the first place.

      1. True, but now they know something they didn’t know before. “Next time we do this, we need better instrumentation, better testing, better redundancy, a real file system,” and so on.

        True, they could have read this all somewhere. There was obvious (to us) process failure before it ever left the building.

        So it didn’t complete it’s science experiments and reach its design goals. So what?

        Just because their success wasn’t where they were looking for it doesn’t mean they didn’t succeed.

        1. They wasted the cost of the satellite, and if they had to pay, the cost of the launch. They may have to wait for another launch space if they try again.

          I always thought Sputnik was simple because they actually were learning from it. Don’t invest e lot for the first try. Same with OSCAR 1 in Dec of 1961. Back then they did learn from each launch, a brand new thing.

          But that was fifty years ago. A lot of detail has been worked out from those pioneers. A design can fail, but it’s harder to claim “oops, we didn’t think of that”.

  2. In case others are curious about space debris issues as I was…

    “CubeSats launched inside pressurized cargo vessels and released outside the International Space Station are of little concern to space debris experts. The space station orbits at an altitude of about 260 miles, or 420 kilometers, where aerodynamic drag from the outer wisps of Earth’s atmosphere often brings CubeSats down within months.”

    https://spaceflightnow.com/2015/07/30/nasa-tracking-cubesats-is-easy-but-many-stay-in-orbit-too-long/

  3. So, they lacked an ADC to monitor their main source power, a brownout detector and a PMIC for their microcontroller. They decided not to test their radio stack by unplugging one of the redudant radio. They didn’t use any kind of journalling filesystem, yet alone one with Reed Solomon correcting codes for SEU that are known to happen at this altitude, and are usually very well feared on rad tolerant hardware (which they didn’t even use).

    One in one, it’s a perfect success to demonstrate that even the most basic rules in electronic development are required. I don’t understand how they were allowed to board a rocket. All of those defects are detectable on Earth, and wouldn’t even be allowed in a v2 prototype here.

    1. If you read the text carefully, you can see that some of the mistakes (e.g. radios) were components from the platform that was not developed by their team but (probably?) bought with the 1U. So it was clearly a platform provider fault…

    2. I can’t imagine putting the work into a redundant system of any kind and no one in the group thinking to actually test it. Thats a level of optimism that I doubt i’ll ever have.

      1. No it is not wasted.
        “1U CubeSat developed by students of the AGH University of Science and Technology in Kraków”
        This was a student project. One of the goals was to teach the students. That is what a lot of people on here do not get is that this was designed as a learning experience. Part of being a good engineer is to the ability to look at a failure and learn from it.

        1. But there’s a difference between failure on the ground, and failure in orbit.

          I remember reading in 1971 about an AMSAT test flight, it must have been for OSCAR 6. They took it up in a small plane to try it out. Maybe not comprensive, but at least they tested the radios. It was like a very low orbit satellite.

  4. Not having a ADC to monitor their main source power & batteries, shows a total lack of common sense. That is the second thing you add after the processor & power supply. How could that have been missed. Clearly, they were playing with things they were not ready to do.

    1. dude – they are students. They did a fairly good job, all things considered. You can be smug if *all* if your projects are perfect, but I suspect you have had failures.

      I do mission-critical embedded systems for a living. But – I am a k-krusty old fart with my share of scars. I *know* what to test because of those scars. Plus, I typically have the luxury of time and resources, including other k-krusty old farts wit their own scars. Quote: shows a total lack of common sense /Quote. – No, it shows a lack of experience and the scars that come with that experience.

      They have had a *fantastic* learning experience. Your statement is like criticizing an art student for not being Picasso.

  5. Having been involved in a few student satellite projects, as part of the FUNcube team, it’s typical that timescales are the biggest challenge, students usually need project papers written for assessment in a couple of terms. The paper here mentions development to delivery was one year, which would have been almost entirely focussed on their payload and not much on system design / engineering, they chose to try and buy that in from SatRevolution and probably had very little ground testing time too. Then, if you want a launch to write up, you go anyway.. It’s a shame about the lack of sensors in the bought in parts, the minimal power budget margin and the poorly considered charging behaviour (no hysteresis, or low voltage safety modes) that prevented them from operating their payload – I don’t know if these are configurable features of the commercial platform, if not then some product improvements are possible. They made a good choice to use SatNOGS network to obtain maximum telemetry and gather some learning from the failing device!

    The ‘Crucial Errors’ analysis section of the paper is pretty decent (ignoring a comment about lighting conditions in space!). Professional teams have failed more spectacularly before now – I would be delighted to see this reattempted after addressing the issues, as a ferrofluid flywheel design would be very useful.

  6. when I read that it was students, and the mistakes they made, I assumed it was early high school..
    The lack of testing on the ground was breathtaking. So was the lack of understanding of designing anything ie nothing monitoring and managing the batteries in a project dependant on solar?? – this project would have failed for being on someone’s farm let alone in orbit.

  7. A maxim for ALL “student” space development projects is that “you can’t go up and fix it”.
    This should be plastered across all lab walls, computer screens, documentation and inside the eyelids of each participating “student”.
    This is a basic engineering design principle that seems to have been lost

  8. What impresses me is the unsparing self-examination and frank disclosure of lessons learned. That paper should be required reading for young scientists, for style as well as substance.

  9. CubeSats and Student-led engineering projects are created so that failure is an acceptable outcome. I 100% agree with the commenter above that said their successes are due to their scars, which are due to their mistakes. I would hire any one of these students over many “mid-career” professionals, because they have done something hard, and they have learned from it.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.