Whassup with the Aurora HDR app sharpen halo?

Still a week 2 noob here, but going through some old HDR work to learn Aurora HDR 2019. I had an old series of HDR images from Sebago that was interesting: Boats on a dock. Boats that move constantly even on a calm day, and thus are very difficult to deal with.

The original base images are here:

base-1
base-2
base-3
base-4

I used hugin to align the images and produce the following HDR image shown below:

Hugin produced HDR image

I thought this was amazingly good. You can see the blur artifacts from the movement of the boats (the number 15; the motors, etc), but if you don’t look too closely this is a nice image.

Below is the attempt with Aurora HDR 2019, Version 1.0.0.2549. I imported the base images with Auto Alignment, Ghost Reduction, and Chromatic Aberration Reduction all turned on. Then I turned off all the Filters. (Meaning this is not going to be the final result.)

Aurora version

Obviously a fantastic job with the blurring. Amazing. I can work with this.

Except I can’t. Something is causing sharpening halos. I’ve tried turning all the filters on and off, but even with everything off, nothing eliminates those halos, afaict. So where did they come from?

Here is closeup of the problem: Look at the sides of the posts. Hugin on the left shows that it is possible to combine the images without excessive halos. Aurora on the right show unacceptable halos.

Not all my images show excessive halo. I did a series of work with a much better camera at http://www.clevercaboose.com/2019/05/07/tablerock-trip-20190414/ But now I needed to stop and figure out what is causing this.

I created a support ticket with Aurora. Quick response:

..it’s quite normal behavior of the software…the RAW photos you’ve sent to us are quite low-resolution ones, and they come out aligned pretty well considering their resolution and format. You might have achieved a better result with RAWs though. The halos you may see are not halos, but light on the photo increased by our tone-mapping powered by AI which increases the contrast.

Contrast that looks like a noob that just discovered the unsharp mask filter! I’d rather a more adult AI.

A note on the word “halo”. Specifically I am talking about overshoot and undershoot.

But perhaps the problem IS low resolution. These ARE old photos.

Try again with 2019 quality images:

hidef-base-1
hidef-base-2
hidef-base-3

Here is an HDR composite with ALL Aurora filters turned off.

ALL aurora filters turned off

The light on the horizon IS “haloed”, excuse me, AI-powered contrasting. But there is also contrast from the original images. Has Aurora added to this even with all filters turned off? Below is an area on the horizon zoomed in:

Zoomed in horizon

If there is any halo,. it is present in the originals as well. I have no cause for complaint. Aurora is excused.

Caveat: Play around with the filters (in particular the “HDR Smart Structure”) and you will find many ways to create crappy sharpening. So don’t turn the knob to 11.

So the issue is resolved. But be careful working with low resolution images. The AI isn’t very smart all the time.

Robyn’s last game with Mike Cleary, Feb 18, 2018

Today was the last game of 10 years of my daughter playing for this man. When she started she could outrun anyone on the field, but a gust of wind might blow her over. A gazelle on a top team in the city. She got to travel far away to play in miserable weather. She got to learn that she is deeply competitive, even ruthless, but never dirty or unfair. She got to learn team success. She got to make mistakes that cost the team the tournament. She got to sit on a bench and wait for another chance. She got another chance and failed at that too. She got to endure two frustrating years playing only 1/3 of a game on a struggling no longer top team in the city. She got to learn that if failure is one’s fate, then not complaining and continuing to work hard every day is how you beat it. That character and perseverance is necessary to win in the long run.

She got to learn that she is a Marine by nature: That she’ll take that hill or die trying. Anything less is unthinkable. And that most people are unlike her, so she’ll probably spend much of her time climbing that hill alone. But that the view is worth it.

She got to learn that life is unfair. But that even if one is too small to hold onto the ball, one can always develop a great first touch so as never to be caught with the ball in the first place. Which now makes one a great passer. So weakness overcome at one stage of development becomes an advantage at the next. And that when height and strength finally do arrive, payback is beautiful.

The gazelle is now a panther.

I didn’t always have faith my daughter was going to be a good player. Those failures were painful, and it was my first rodeo. She had athleticism, but lacked skill and received no reward. I didn’t think anyone else had faith in her either, perhaps even her coach. 5 years in, after another disappointing year, I asked if we needed to move on. He was shocked at the question, answering: “Absolutely not. She is about to become very good and I have big plans for her.”

(What he only told us later was that he was alone in that opinion. All the other coaches had recommended he drop her.)

That meeting ended our concern. I trusted the man. That summer she grew, and that fall everything changed for the better.

3 girls who played all 10 years together. 8 who played the last 6. That just doesn’t happen. You have to be incredibly lucky.

Lucky to have a coach powerful enough to win the internal club battles and keep the team together. Who has always done the right thing for the player, as opposed to himself. Who cares about character and chemistry and playing soccer the right way. Who praises a loss played well, and criticizes a win played badly. Who criticizes without belittling. Who believes that fundamentals cannot be taught via shortcuts. Who demands you play it out of the backfield. That you pass to build up an attack instead of just kicking it downfield. Even if it means you lose for a few years. That you need to take that hill the way he tells you to take that hill, and if you die today while trying, today isn’t the most important matter.

Where you are in 10 years is the most important matter.

Thank you, Michael Cleary. Your character matters, and that is why this team always stayed together. My daughter has been incredibly lucky to have you as her coach. May you have many more teams of quality, and may you be respected and rewarded for being the great coach you are.

Kitchen remodeling before and after

So we’re remodeling the kitchen. The impetus was getting fed up with the electrical stove and lack of vent. That plus watching the excellent contractor working next door and his crew showing up 7AM to 7PM including weekends. Hard to find good people so we pulled the trigger.

Expected to take 1 month. Removing the wall to the dining room. Hard wood floors. New appliances except for the fridge. Custom cabinets.

Following picture was taken the night before the work started, Jan 21.

Kitchen the night before

Day 1 was a full day of demolition. Dramatic and happens fast. They even managed to add the frame for the pantry. Jan 22

24 hours later after demolition

Electrical dry walling was next. After that was the hard wood floors. Thus endeth week 1, day 4, Jan 25.

Day 5. Electrical, dry wall, and hardwood floors.

Next was the custom cabinets. 6 full days of 2-3 people. We had a tricky situation with the bay windows and future sink. No problem. These guys figured it out as they went and did it perfectly. We added a peg board to an open space, and neat triangular diagonal section facing the dining room. Do not buy standard cabinets and expect to like the results. I repeat, do not buy standard cabinets. The customization is critical and well worth it.

Day 14. Custom cabinets completed

Onto painting. And this gets stressful. First, you’re going to come home to a house where you can’t breathe. Meaning the entire area is either walled off or completely open with fans blasting the air out windows into 10 degF outside weather. You bundle up and watch the house temperature drop to 53 degF and do your best.

Worse is that you’ve picked the wrong color and it is overwhelming. The ripe apricot looks bright orange in warm light (see peg board) and coral pink in cool light (see fridge cabinet)! Lesson learned: Do not use water-based paint to test when the final paint will be oil-based.

Day 18: Ripe Apricot looks like orange and coral pink!

So we cost ourselves a day. We didn’t want to go into the brown-ier shades, even though that would have been safer, given the different lighting conditions. We tried the next darker shade of ripe apricot and that did what we wanted. Even that was scary, as practically the entire kitchen was one dominant color, and we had to take to visualize and gamble that the walls and countertop would balance things out.

Day 21: Corrected color for cabinets. French Vanilla for the walls completed.
Day 22: Cabinet doors replaced. Neutral/warm lighting, early morning sunlight. Still a balance of warm (left) and cool (right) lighting

And next is the backsplash. We started off wanting something plain and simple that would match the wall color. But plain tile disappears. So we thought we need smaller tiles. That was better, but still not quite there, see below.

This wasn’t bad backsplash.

Also, prices range from $3-80 per square foot. We also had to consider that black pearl countertops would change the balance as well. It came down to the two options below:

Final two choices in backsplash options. We decided to go bold with the tile on the right, as it should go better with the Black Pearl countertop.
Mixing up the lighting

Construction got behind 1 day and then a 2nd day because of bad weather and someone getting sick. On the day they were supposed to put in the countertops, no one showed up. I went shopping for lights at 7PM and when I got home three men were fitting the countertops, grinding granite in the driveway and fitting the new sink. One enjoyed his 0% alcohol heineken. All done by 10PM.

Countertop and sink installation, day 24

Shortly after we got the big stuff in. And then I had the epiphany of getting some art work from our good friend Dierk Van Keppel at
http://www.rockcottageglassworks.com/ . And then we had a functional kitchen and things slowed down a lot. I got obsessed with wifi lighting and invested heavily in the Lutron Caseta system. A premium price, but only by 10% and they are the best. Electronic stuff that just works is worth a premium. Took some time to get drawer handles in. Bought back the cabinet people to add some internal shelves on rollers. And all the little loose ends.

Fast forward to March 23, or 2 months since we started. 98% done and all is fine. The color grabs your attention and changes throughout the day. I’ve been trying to create a good HDR image and failing so far. So here is the way it looks when fully lit:

Near completetion panorama. Bright

And a little darker

Near completetion panorama. Darker

Dr. Robin Richard MacDonald

Dr. Robin Richard MacDonald, died peacefully just shy of his 83rd birthday in Leawood. He is survived by daughter Helen Carson (Jeff), and son Richard MacDonald (Pearl) of Kansas City, and grandchildren Robyn MacDonald, Winston Mac-Donald, Grace Carson, Emma Carson and Jacob Carson. He was preceded in death by his wife Heather Glee two years ago, and parents Hector and Francina.

He was born in Bloemfontein, and grew up in Port Elizabeth, South Africa. After attending U.C.T, he began his residency in Radiology at Groote Schuur Hospital where he met a Head Matron named Heather Glee Curtis. His studies and teaching took him to universities in Durban, Melbourne, Adelaide, and London, where he was a Fellow of the Royal Faculty of Radiologists in Great Britain.

After immigrating with the family to America in 1977, he was on the staff of the Cleveland Clinic before moving to Kansas City, where he practiced at Baptist Memorial Medical Center until 2003. He then joined the St. Luke’s Hospital Plaza team. From reading the letters from colleagues and patients that he kept, his dedication to medicine and his compassionate care of patients was the cornerstone of his purpose. One letter from the Director of Radiology in Adelaide noted ‘he could be diplomatic or forthright as required, but seemed to have the happy knack of knowing when either was called for’. Upon occasion, his students and children might recall the latter!

He grew up loving sports, and captained the cricket team for U.C.T. He played 12 first-class international cricket matches in the 1950s, representing the SA Universities team against Australia, New Zealand, and the West Indies. In 1956, he scored a decisive 29 in the final game to win the Currie Cup for Western Province.

After the loss of his leg at age 26 as a result of a blood clot ended his cricket career, he began a lifelong love of golf. He twice reached the semi-final of the Natal Province Amateur annual competition with a 2-handicap, scoring an albatross. In South Africa, he was a three-term President of The Royal Durban Golf Club.

He eagerly adopted all American sports with equal zeal, and relished a full weekend of sports with the family over to enjoy golf and the KC and KU teams. In later years, he was avid follower of the PGA.

After initially retiring at age 65, he joined the St Luke’s South Goppert Center for Breast Care. He was passionate about providing dignified diagnosis for appreciative patients. He enjoyed a very happy decade of working 6 days a week in the morning, playing golf and gin rummy with friends as a member of Indian Hills Country Club, and attending grandchild academic and athletic events with love and quiet pride.

‘Oupa’, as he was affectionately known by his grandchildren, was a kind and generous man, devoted to his family. His quick wit, big heart, and bad jokes were legendary, and his friends and extended family remember him as honorable above all else. A soft touch for all animals, the MacDonald Labradors knew they had hit the jackpot.

After caring for Heather, it was his turn with Parkinson’s. Robin went peacefully in his sleep to be reunited with his beloved wife for their 59th wedding anniversary.

Honoring his wishes, he will be lovingly remembered at a family memorial service. Donations can be made to St Luke’s South Goppert Center for Breast Care Fund at St Luke’s Foundation, 4225 Baltimore Ave, KCMO, 64111.

Obit at kccremation

KCStar

I added the following as a personal note when I shared this to facebook:

I have never forgotten for a second how lucky I was to get my parents. I never saw them argue. And I never won a single argument because they just were never wrong. My Dad was gone for months and years at a time preparing to leave the country. And when he was home, wherever we went he ran into a good friend. He knew everyone and everyone knew him. A man who filled the room and made people laugh. My favorite memory is of him giving the annual New Year’s dinner speech at the club with everyone in the palm of his hand! I got to hang out in the parking lot afterparties with the world’s best rugby players and cricketers. We were big fish in a really nice pond.

Life was pretty good when we came to States as well. We had season tickets to the Royals in their prime, froze our asses off in the nosebleed Chiefs sections, and spent all weekend watching sports on TV, and playing golf together. I never had a fight with my parents. I just got to listen, observe and learn. Lucky lucky lucky!

When the kids left home, my Mom become a world class dollmaker, as well as a gardener. My Dad continued to play golf and gin rummy, and loved his work. My God, doctors work hard! Gone by 7 in the morning. Home at 7 at night. 3 nights a week on call. Working every Saturday. Week in week out. He knew his medical victories and died every time he made a mistake. He finally “retired” so he continue to work “half time” on his own time. Never again to feel the pressure of the clock. For the next decade he was gone every morning, finishing early afternoon, and hanging out with the boys at Indian Hills until the evening. His reward for a lifetime of commitment to others without complaint.

My Mom’s cancer diagnosis forced him to quit the next day and figure out caretaking. And that was the beginning of the decline because he was such a social animal, and my Mom hated to be dependent and know what it was costing him. Still, they made the best of it, as one must, and only asked for help when it was absolutely necessary. And it was also the start of my nightly calls to my Dad which lasted anywhere from 10 minutes to 3 hours depending on how interesting the subject. The most popular subjects were in-depth analyses of something his grandchildren were going through. Other times it was simply a laughing “Pretty good day. Nice to not have anything needing to talk about.” Every once in a while it was tech support for his computer, which was so infuriating I always eventually just gave up and drove over there to fix it.

Both my parents had great lives. As medical people, they had zero bullshit tolerance about their ends. And those were good too. You just can’t get luckier than that.

Heather Glee MacDonald Obituary

Heather Glee MacDonald, 86, died peacefully on June 16, 2016 in Overland Park, KS.

Born in Keetmanshoop, South West Africa, she grew up in Cape Town, South Africa, and was a Nurse. Becoming the youngest Head Matron ever at Groote Schuur Hospital, she inspired many of her generation to follow in her profession by the compassion and devotion shown her patients. She is survived by her loving husband of 56 years, Robin, daughter Helen Carson (Jeff), son Richard MacDonald (Pearl) of Kansas City, and sister Meg Colleen Arter (John) of Cape Town, South Africa. She was preceded in death by her parents Tom Macready Curtis and Elsie Iris Willmot, sister Elizabeth Donlyn Perry and sister Cathleen Nella Wagenfeld.

She leaves behind a legacy of devotion to family and the grandchildren she loved so dearly – Robyn MacDonald, Winston MacDonald, Grace Carson, Emma Carson and Jacob Carson. After arriving in Kansas City in 1978, she was active in the community helping those who are most vulnerable and under- served, a passionate advocate of justice. She enjoyed a lifelong love of nature, gardening, sewing, in time becoming a prize- winning doll maker, and relished staying in touch with the family that had spread around the world. She is remembered near and far for her marvelous sense of humor and her gentle touch for all animals, especially the MacDonald Labradors who knew they’d hit the jackpot! Her kindness and compassion, grace and bravery are an inspiration to all she touched.

Honoring her wishes for a private family service, she is lovingly remembered for a life lived with dignity and purpose. Cremation Society of Kansas & Missouri, 8837 Roe Avenue, Prairie Village, KS 66207, 913-383-9888. www.kccremation.com

Published in Kansas City Star on June 22, 2016

Spatialite DLL hell

Spatialite is a geospatial-enabled extension to sqlite. I must have it. I intend to use it inside our commercial java product, which has windows and linux users. The development environment is Eclipse. The OS is Windows 10.

The author of spatialite mentions that the windows environment suffers from DLL-Hell. Well yes it does.

Installing sqlite and the xerial-jdbc software is easy and not covered.

You download the spatialite files here. It isn’t well explained, but you need the “mod_spatialite-.7z file.

My preferred eclipse setup is to create a /lib folder in my eclipse project and put the jars and dlls there. Below is a simple test project showing the locations of the files. Note that the sqlite-jdbc.jar is included with a “Native library location” that points to the folder containing the dll files. This is the standard way to do things. It works most of the time.

The author of spatialize has some good installation instructions and sample code. I added the sample file and tried to run it.

The result was the following error:
Failed to load extension mod_spatialite. Trying to find file manually
Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: C:\apps\eclipse-64-workspace\spatialite-test2\lib\mod_spatialite.dll
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1827)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at SpatialiteSample.main(SpatialiteSample.java:40)

This is the well-known dll hell. The eclipse launcher uses the “root” folder of the project as the root java location. Here are the details. The command line is:
"C:\Program Files\Java\jdk1.8.0_121\bin\javaw.exe" -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:57143 -Djava.library.path=C:\apps\eclipse-64-workspace\spatialite-test2\lib -Dfile.encoding=Cp1252 -classpath C:\apps\eclipse-64-workspace\spatialite-test2\bin;C:\apps\eclipse-64-workspace\spatialite-test2\lib\sqlite-jdbc-3.18.0.jar SpatialiteSample

At this point, someone gets desperate and recommends adding the spatialite path to the windows PATH variable, or drop the dlls in the /system folder. But this is intended as a professional installation, not something for the enthusiatic developer. These are not acceptable recommendations.

To simplify the problem, a good next step is to move the dlls to the root folder. Can’t miss them there, can we?

And that works. The “mod_spatialite.dll” is found in the root folder. The command line is:
"C:\Program Files\Java\jdk1.8.0_121\bin\javaw.exe" -agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:57854 -Dfile.encoding=Cp1252 -classpath C:\apps\eclipse-64-workspace\spatialite-test3\bin;C:\apps\eclipse-64-workspace\spatialite-test3\lib\sqlite-jdbc-3.18.0.jar SpatialiteSample.

So we can run spatialite in eclipse if we are willing to place the dlls in the root directory. We are not.

At this point, I am extremely puzzled. To be continued, hopefully…

Java Null Pointer Static Analysis in Eclipse – JDT vs CheckerFramework – False-Positive comparison

We’re talking about this and this. For those of us who program in Eclipse using Java, what is the best tool(s) for checking null pointer exceptions.

There have been several attempts before, but nothing has made its way into the official language. Eclipse does provide good null pointer checking out of the box, but there is only so much it can do. To improve the power of the checkers, we need to add annotations.

Note: In the following, I am using JDK 1.8.

1) We have Eclipse itself, using the JDT annotations Eclipse help.
2) We have the standard checker framework.
3) We have FindBugs, which I believe is used by sonarcube.

There is great power to have something built into the environment, so the standard eclipse system has a strong advantage. Unless it has serious problems, it should be part of the standard development environment.

Unfortunately, it does have serious problems. Take a look at the following program:

public class TestNullCode {

    @org.eclipse.jdt.annotation.Nullable
    private String testJdt;

    public void testJdt() {
        if (testJdt == null) {
            testJdt = "not null";
        }
        // this line causes a false-positive warning
        // Potential null pointer access: The field testJdt is specified as @Nullable
        if (testJdt.equals("not null")) {
            System.out.println("will work because testJdt is not null");
        }
    }
}

This example shows a false-positive that is provably incorrect. The compiler issues a warning (or error, depending on your settings) on line 12, that is untrue.

Frankly, this is pretty sad. The reasons are justifiable, but the result is sad, nonetheless.

For non-final @Nullable fields the simplest strategy for avoiding any nullness related risks is to pessimistically assume potential null at every read. This means for strict checking no flow analysis should be applied to @Nullable fields..

How does the checkerframework do? Answer: No problems at all:

public class TestNullCode {

    @org.checkerframework.checker.nullness.qual.Nullable
    private String testCheckerFramework;

    public void testCheckerFramework() {
        if (testCheckerFramework == null) {
            testCheckerFramework = "not null";
        }
        // this line is ok
        if (testCheckerFramework.equals("not null")) {
            System.out.println("will work because testCheckerFramework is not null");
        }

        testCheckerFramework = null;
        // following line is flagged by checker framework
        // dereference of possibly-null reference testCheckerFramework
        System.out.println("will fail because testCheckerFramework is not null" + testCheckerFramework.toString());
    }
}

How does FindBugs do? Answer: No problems at all. Same as checkerframework , so I won’t post the identical code.

Eclipse has a workaround that costs a line of extra code:

While this appears to be a very drastic restriction, the remedy is quite easy: before dereferencing a @Nullable field it has to be assigned to a local variable. Flow analysis is then safely applied to the local variable with no risk of side effects, aliasing nor concurrency, since local variables are not shared with any code locations that would be outside the scope of the analysis. I.e., the flow analysis can see everything it needs to consider regarding local variables.

public class TestNullCode {

    @org.eclipse.jdt.annotation.Nullable
    private String testJdt;

    public void testJdtCopy() {
    	String copy = testJdt;
        if (copy == null) {
        	copy = "not null";
        }
        //This is fine
        if (copy.equals("not null")) {
            System.out.println("will work because copyis not null");
        }
    }
}

I consider this “solution” ugly. I get it, but I don’t buy it.
At some point, I would expect the Eclipse engine to get better.

So at this point, we still need the other tools.

You can download my example eclipse project here.