If a person owns something like a vehicle, there may be a number of interests in the vehicle: appearance, performance, safety, cost, longevity, freedom, distance, weight capacity, space, maintenance, fuel efficiency, clearance height, traction, etc... Many interesting factors to consider. However: ego, honesty, love, faith, trust, patience... these are not factors of the vehicle. The vehicle is physical. If the vehicle has an autopilot, then I'll make use of it, but it is not the ego. It is better to take responsibility. If my vehicle is prideful, dishonest, unloving, uinterested in having faith in others, untrusting, impatient, etc... then what do you think the cause is? I am. Ego Ami. I am not the vehicle. If I am driving it though, then I am responsible for its behavior. If I tailgated you, failed to let you in, wouldn't stop for you, didn't forgive you when you ran the red light, then I suspect it will not be the vehicle itself that was the cause. There will be nothing in the vehicle to kill to change that behavior. I will not be gouging out the headlights, disabling the computer, removing the fuses, starving the vehicle of fuel, or parking it for long periods to find enlightenment. Punishing the vehicle won't help. If I run someone over and kill them, then it won't help to sentence the vehicle to life in prison. The vehicle seems impermanent, because every vehicle eventually gets dismembered. Every day is a day closer to death for the vehicle.
Yet according to physics: while the vehicle seems impermanent by what we can see, it is not impermanent. An account of everything that I do with the vehicle is being kept by the vehicle and the physics itself, in a permanent manner. Why, or for what purpose, I don't know. My behavior with ego, honesty, love, faith, trust, patience, or the negative thereof, are on that permanent record which you have so decisively called impermanent. I don't know how it matters, but I learn a little every day.
At the same time my behavior with my ego, honesty, love, faith, trust, patience, for example, or the negative thereof, can be altered. Somehow, and again I don't know how, I am not permanent. While I made some poor choices in the past with my vehicle, upon both interaction and reflection I (we) found it beneficial and possible to change. I have to accept that other people can somehow change their character. If I am permanent: once a sinner, always a sinner, then I should be destroyed for being faulty. I have sinned and been at fault in the past. People are not permanent, but can somehow change from dishonesty to honesty, from hate to love, from unfaithful to faithful, and go the other direction too. To love can be to consider the character of someone impermanent, for you might not love the character, but you can still love the person. I cannot change the past, but I can change my behavior in the future. The Buddha's concept and valuation of impermanence is either misguided, or misinterpreted.
Yes we can meditate, we can joy ride, and drive only for our own selfish pursuits. I have done these things too. A bit of alone time, quiet time, and focus time can be good, but it is the together time where we discover ego, honesty, love, faith, trust, patience, for example, or the negative thereof. As we drive on the road with each other, and for each other's purpose, then we get to see and know each other's spirit. On a deserted island you won't get to see your honesty or dishonesty, will you? Meditating, do you think you can see your dishonesty? As we contend with the ensuing turmoil that we see, what we see we can somehow keep or change. If we seek isolation instead of the turmoil of spending time together, then we do not see, and we do not learn about our spirit. That has been my experience with myself and the people I have driven with.
Answering your questions: Does the state of something impermanent really matter? Yes, because seeing the state allows you to choose whether to change it. Also, I find that what you are ultimately calling impermanent: history, is actually permanent.
Does your expectations of this place change anything? If we choose to change a place, then it will be changed. If by this place, you really mean another person, then no... I cannot change another person, but I can set an example for them and help show them what I see in them so that they may choose whether to change. Vice versa the same.