Highly distributed, autonomous and self-powered systems operating in harsh, outdoors environments face several threats in terms of dependability, timeliness and security, due to the challenging operating conditions determined by the environment. Despite such difficulties, there is an increasing demand to deploy these systems to support critical services, thus calling for severe timeliness, safety, and security requirements. Several challenges need to be faced and overcome. First, the designed architecture must be able to cope with the environmental challenges and satisfy dependability, timeliness and security requirements. Second, the assessment of the system must be carried on despite potentially incomplete field-data, and complex cascading effects that small modifications in system properties and operating conditions may have on the targeted metrics. In this paper we present our experience from the EU-funded project ALARP (A railway automatic track warning system based on distributed personal mobile terminals), which aims to build and validate a distributed, real-time, safety-critical system that detects trains approaching a railway worksite and notifies their arrivals to railway trackside workers. The paper describes the challenges we faced, and the solutions we adopted, when architecting and evaluating the ALARP system.