In order to provide adequate assistance to cognitively impaired people when they carry out their activities of daily living (ADLs) at home, new technologies based on the emerging concept of Ambient Intelligence (AmI) must be developed. The main application of the AmI concept is the development of Smart Homes, which can provide advanced assistance services to its occupant when he performs his ADLs. The main difficulty inherent to this kind of assistance services is to be able to identify the on-going inhabitant ADL from the observed basic actions and from the sensors events produced by these actions. This chapter will investigate in details the challenging issues that emerge from activity recognition in order to provide cognitive assistance in Smart Homes, by identifying gaps in the capabilities of current approaches. This will allow to raise numerous research issues and challenges that need to be addressed for understanding this research field and enabling ambient recognition systems for cognitive assistance to operate effectively.